Fog Computing and Ioe(revised)

read more

Paving the way to IoE. 

General Fog Computing: 

	Structure: 
		-Dynamic Apps
			
			Fog servers examine traffic to determine when an app is being used sufficiently enough to request a virtual version. When the time comes for a virtual request to be made, the Data Center responds with the CPU/RAM requirements required per user as well as where the data 			should reside for the app. We break down a dynamic app, or really any app into a few components: app logic, database tables (data), static content (HTML templates, CSS, JS, etc), and user preferences. Regardless of the dynamic app, the app logic, static content, and user 			preferences should be able to be loaded onto the Fog without concern, the problem is with the tables/data. Depending on the nature of the app, all, some, or none of the database tables should be sent to the Fog server. For relatively small apps where dynamic data is shared, all 			data could be sent, and the Data Center orchestrates sending and receiving updates to and from existing Fog servers. The only addition that needs to be made for these kinds of apps is an interface through which updates can be received and sent (and appropriate changes made). 			Apps with user specific tables behave similarly. Both of these would result in greatly reduced latency for the user. The last is where no data is held on the Fog, and all communication with the database must be done over the network (again, this requires programming an interface 			for sending/receiving and implementing updates). This options doesn’t provide any/much benefit in latency, but it does offload work from the data center. As mcany users would be using this app at one time, batching of network requests can be done to reduce network utilization. 			Inside a fog server, the app logic and static data is shared amongst users with each users data tables/user preferences siloed off from the rest of the machine. When the number of users/resources goes below a certain threshold, the VM would be destroyed to make room for other 			apps. 	
		
		-Static Apps
			-Static apps would be those that are entirely self-contained, and don’t need to talk to the Data Center periodically for updates. These apps work beautifully with the Fog Computing concept. 

		-Real Time Video / Gaming:
			-The distribution of Fog Computing is what helps with this? Say two people on the East coast are Video Chatting/gaming, rather than routing packets through a Data Center in California, a Fog Computer in Virginia could be used. 
			-Skype behaved very similarly to how Fog computing would behave. http://www.mjalali.com/blog/?p=10. They have super nodes which are Skype clients with sufficient CPU, Memory, and Bandwidth to act as relays for checking if users are online, not for routing traffic. 

			Questions: 
				-How would Fog help with this at all? I don’t understand. 
		
		-Geographically Distributed:
		
		-Application Assistants: 

		-Standalone Applications: 
			Applications that don’t need or shouldn’t have or can’t afford a Cloud could be distributed on the Fog. 
	

		-Static Website/Content
			-HTML Content: this is simply caching the HTML on the network device. Security is not as much of a concern here. Essentially, the network device would be able to unpack packets to examine the request, determine what page it’s going to, add to an 			increment var for that site that indicates how many active users have tried to access it, and then if the page is in its cache, it’ll serve up the page itself. There are a few things that surround this, such as having the website host specify whether they want fog caching 			enabled, as well as specifying an expiration date for any pages that do get fog cached. 
			
			-Downloads/recorded video: Downloads are going to require a lot more space on the fog server’s HD, but may provide a huge benefit to end users. 
	
			Questions: 
				-Should the fog server by default cache the web pages, and check to see if they should turn it off, or should it be disabled by default? 
				-Each fog server would configure its settings for thresholds on when to cache downloads/HTML content. 
				-Because the Fog would have the relevant data in the data center, and because there are fewer routers between the Fog and the end users, would the use of Multicast be more likely possible?
				-Is Jitter a concern with the implementation of fog servers? You don’t want to have some web pages be very quick to load and then others be quite slow. 
				-Should the fog servers tell the cloud for every req.uest they answer on behalf? Just to keep public record. 
				-How will DNS resolution work for Fog computing? Traffic still addressed to main server, but this router will just intercept the traffic. 
				-How will Fog Computing defend against man-in-the-middle attacks? 
	
		-Comments:
			-Something like VMWare’s vSphere could be used to run multiple virtual machines on a single fog computer. 
			-ISPs could have deals with certain companies to allow their data centers to request a VM sent down to a Fog server, and then the remaining idle compute/storage could be used for general applications on the internet, and the ISPs would send that cloud server a request for a VM if the network 				load is high enough. 
			-LinkedIn (and apparently other web companies), setup PoP (Point of Presences) which are just small scale data centers which act as endpoints for TCP connection requests. They don’t have compute, but they could and this would be essentially the start of a Fog layer. 
			

To truly profit from the benefits derived from data center consolidation and virtualization, IT managers strive to virtualize different layers within the data center, mainly the server and storage layers, using technologies such as VMware’s vSphereTM or Microsoft’s Hyper-VTM. The rate of server virtualization depends on the number and criticality of the data center’s applications, meaning most IT managers will usually start with virtualizing the less critical applications and only once they are all virtualized, move to the more critical applications. 
reason to do fog:  because of some of the costs around linkages for cloud upload."
The vendor earlier this year unveiled IOx, a mash-up of Linux and its own IOS operating system specifically designed to run self-contained applications on networked devices.


	General Questions/challenges: 
		-Good paper: http://conferences.sigcomm.org/sigcomm/2012/paper/mcc/p13.pd, but they’re thinking about Fog only really how it’s going to support IoE. Fog can come into play today through ISPs as a way to reduce their bandwidth and speed things up for users. 
		-Should you allow the cloud side to request its VM to load on fog devices? Or should only the fog device be able to start conversations?
		-What to do with all the idle processing? 
		-What’s the incentive for networks introducing fog servers? Reduced bandwidth consumption. 
		-What’s the security look like for any App running on a fog server? 
		-Net neutrality for Fog computing? Tony brought this up. What about the owners of those compute machines saying “hey, we’re doing all this processing for you, why don’t you pay us something 		for better faster speeds?” 

	Thought provoking questions:	
		-What are things that were impossible due to excessive/uncontrollable latency? 
		-Some Cloud processes/data analytics were impossible because it was infeasible to send TB of 		unimportant events for processing, Fog allows edge filtering. What are these applications for users?
		-What are physical/geographical events that require time sensitive feedback?

	Other notes: 
		-Big data comes in here because you could predictively load apps in locations. 


Fog Specifically for Home: 
	
	Structure: A home would have dozens of IP-enabled devices, all of which may send data upstream to the device creators for analytics/diagnostics. That data should be filtered at the network edge and only important data should be sent to the cloud. Applications could also be created for the router to increase the customizability and decrease the latency of interaction between devices (Currently, this interaction is done through the Cloud for Nest). This is an application that’s inherently distributed and is better installed on home routers than put in the Cloud. 

	Nest: This relies on the Cloud to interact with other devices. You interact with a web API for a device to set temperatures, view temperatures, turn on fans, etc. 

	HomeKit: Homekit is essentially a controller itself. It organizes all devices that it can connect to into homes > rooms > accessory > characteristics. Nest would probably just be enabled with HomeKit so that it could be used by apps. Essentially, apps from the app store would be IFTTT rules, because the apps are the logic that interacts with the devices. 

	Could look into SmartThings Hub

RockWell Automation: 

	Capture: only one layer is involved in capturing the data, and that’s the network’s very edge. One device that needs to be monitored has some number of sensors on it. Depending on design decisions/space availability, etc. the company may have a router with compute (no storage unless application requires it) for each device or one connected to many devices. Upon setup, interfaces on the router are configured to recognize certain events. So, for setup one could have a very simple interface that covers the types of events one could want to monitor, such as threshold breaches (the incoming value went below/above a min/max) or boolean events (such as the device “is moving”). Other settings to configure here could be polling time, which could be how much of the sensor data to actually listen to. Not every interface must recognize an event, and if it doesn’t then it’s going to be doing straight data collection probably for logging. 

	Sanitize: Sanitization, or selective attention, occurs at each layer of the network. For each device you have the data input, which could be direct from the device in the case of the lowest layer, or you could have the data passed on from lower level devices in the case of layer 2 and above devices. Each device has three planes, the data plane, event plane, and action plane. The data plane is simply the area of the device where the data is coming directly in. Each data stream goes through the preprogrammed event plane, which will determine if events occur. If an event occurs, an event is fired. Actions in the action plane are subscribed to the firing of events. One action may listen to multiple events, and one event may be listened to by many actions. The actions are also preprogrammed and based on the events the action listens to, it may respond by sending different responses. A response could be changing the log frequency, change behavior/state of device, request part replacement, or pass data up. Preferably, for UPIC, events and actions would be put in place that monitor for all events that RockWell would be concerned with, and the event would be as simple as always passing up the event up to the next layer then to the cloud and then to Rockwell. 

	Transport: Transmission occurs over IP. Through a hierarchy which varies depending on the application. 




Enterprise IoE: 

	Structure: 
		-A hierarchy is certainly necessary for effective complex event processing. Small, single task routers with compute would be responsible for a single data stream configured to look for specific threshold breach events, boolean 		events (like “machine is moving”), etc. Each of these routers would be connected up a level, to event recognition routers. The event recognition routers would processes the event data in real time, determine if an event that 		requires a response is occurring, and then send an appropriate response/pass data up to a parent. You split events up into system-dependent and system-independent actions/events: system-independent actions/events are 		those that don’t need to know anything about its neighbors in order to trigger a response. System-dependent actions/events require a more comprehensive knowledge of the system in order to trigger a response. The 		hierarchy allows for 	layers of dependency to exist (you could have localized event dependency, such as a manufacturing line, system-wide event dependency, such as an entire manufacturing plant, or event global event 		dependency, which could be all plants in a company + data feeds from other companies, weather, news, etc: this would have much higher latency as it would go to a cloud unless data is cached, but could still be helpful). 		Event responses could be finer frequency logging, action events such as telling the machine to change behavior/state or shutdown, requesting a part replacement, or simply deciding to pass the data up to higher levels (likely 		directly to a cloud level, but intermediate levels could receive as well if more immediate response may be needed) for big data analysis. At each point of event processing, there are two components: events and data. Events 		are preset actions that can occur when conditions are met, which necessarily means that things happen that the programmer already knows about. The data component is more about tackling and discovering problems that 		the programmer is not already aware of, and those problems would become aware through global or system-wide big data digestion. 

	UPIC: 

		Structure: 
			-The structure here is similar to how the Enterprise IoE structure would work (stated above). The enterprises involved would have opt-in options, they could opt-in to nothing (which would mean they’re essentially not a 			part of anything), they could opt-in to the data plane (which means they would offer up process-anonymized data in exchange for action recommendations from the higher company, which they would implement 			themselves. They higher company (like Rockwell) would use data from all participating companies to recommend actions), or they could opt-in to data and actions, which would allow the higher company (such as 			Rockwell) to recommend/implement actions using data (I don’t think that this one is plausible because the higher company would have to know specifics about the processes). So essentially, the most common situation 			would be a sharing of the data plane. So security on this would require simply a separation between the event/action plane and data plane. 
		

		Questions: 
			-How will companies feel about not owning their own cloud? I mean this is really just a data sharing cloud so will they be okay with it? Also, currently how many companies outsource 			their cloud infrastructure? According to this doc it’s a lot “Many Enterprises want to focus on their core business and leave the management of their IT infrastructure to Cisco. “
			-How is this cross-business collaboration going to be setup? It seems like it’d be hard to get all companies to work this out. Would it make sense for Cisco to provide it and the companies to pay for that as a service? Of 			course other companies will start to offer the same service, but that’s okay. What about high-security companies who may not trust Cisco to run this inter-business cloud? 
			-Two approaches: should every community have its own cloud? Or should they all feed into one massive one owned by Cisco and Cisco sets up communities segmenting data within that 			cloud? Doesn’t seem like there’s much of a difference. 
	
	Challenge: Coming up with a general model of application for customers. 

	Solution: the model assumes that the customer has inefficiencies or problems that can be solved by more data collecting (seems like this is commonly true). So it starts with “What 	problems/inefficiencies are you having?” or “What goal do you need to reach?” (for instance, Boeing wanted to go from 42 planes a month to 60). In response to their goal, we ask what their 	bottlenecks are (essentially what problems/inefficiencies they’re having in their production). Then, we take it from there with sensors, routers, and data analysis. 




Important Notes from research: 

	CDN essentially already does this for static content. They pay ISPs to let them put their servers with the ISP, and application owners pay the CDNs. But why don’t we cut out the middleman and tell ISPs they can buy these Cisco routers to replace their CDN and get money from websites directly. 

Info on Google’s CDN: https://peering.google.com/about/faq.html

How does Google direct its traffic to its own CDN (called Google Global Cache) which are in ISP networks? 
	
	When a user makes a request to a CDN hostname, DNS will resolve to an optimized server (based on location, availability, cost, and other metrics) and that server will handle the request. What are the methods of doing this? This must require some propagation. You could use Anycast: http://en.wikipedia.org/wiki/Anycast, 


Great LinkedIn Article: https://engineering.linkedin.com/performance/how-linkedin-used-pops-and-rum-make-dynamic-content-download-25-faster?trk=spup_mktg_tlntacq_HA_T_ENG_SU_RUM&sf3506722=1

Distributed Databases (Google): http://www.wired.com/2012/11/google-spanner-time/all/: “When one replica is unavailable, Spanner can rapidly shift to another. But it will also move between replicas simply to improve performance. “If you have one replica and it gets busy, your latency is going to be high. But if you have four other replicas, you can choose to go to a different one, and trim that latency,” Fikes says. One effect, Fikes explains, is that Google spends less money managing its system. “When there are outages, things just sort of flip — client machines access other servers in the system,” he says. “It’s a much easier service story…. The system responds — and not a human.””

We are essentially “Dynamic Traffic Server Creation”: https://docs.trafficserver.apache.org/en/latest/admin/reverse-proxy-http-redirects.en.html For example, Traffic Server can be used as a mirror site on the far side of a trans-Atlantic link to serve users without having to fetch the request and content across expensive international connections.

Distributed Computing (Facebook): http://www.wired.com/2012/08/facebook-prism/

PoP (point of presence), is essentially the start of Fog computing minus the compute.

Security concerns of Network caching. Essentially, you’d know that someone on your network accessed that website. Should you make it so that you don’t know what layer the cache came from?