What makes a network an industrial network?
By D Kowensky, H3iSquared
The internet is the largest collection of networks we know of. It gives us access to gain information from a continent on the other side of the world!
The progressed from only text based messaging to what we know are familiar with when surfing the net.
The internet offers us thousands of services and is continuing to grow with different applications and requirements. The reason this is all possible is due to the transport mechanism of Ethernet, it can cater to transport different protocols over the same medium, even with the same transport mechanism. This is how we are able to browse an internet page, chat to your friends with Voice Over IP (VoIP), view your offices remotely with IP Based CCTV systems etc…
So now lets look at a smaller scale network, looking back at the older days companies had to use a HUB. The HUB did have a certain degree of low level intelligence in the fact he could negotiate speed, regenerate signals, allow devices to communicate through it and offer the intelligence to handle situations for data collisions. This was required due to the fact that a FULL Duplex connection was not yet available so all mediums were only able to either transmit or send packets of data at a time. With a shared HUB environment, since there was only half duplex, if there are 4 devices on a network, each device only has a 25% chance to speak on the network. As soon as we upgrade to 16 devices on the network we see each device only has 6.25% chance to speak on the network. With these types of percentages of chance the device can speak on the network, this would not be acceptable to run any form of automation that requires critical timing since the delay period in this environment would be a minimum amount of time to an unknown amount based on amount of collisions and how busy other devices are. In other words, if ANY device in transmitting on this network, no other device can also transmit without disturbing the original transmitting station.
Once the introduction of Switches with FULL Duplex environments were introduced, the amount of determinism on Ethernet networks was vastly improved since now devices could transmit and receive data concurrently.
Yes, this solved inherent concerns for growth of networks with regards to amount of users and amount of data transfer required.
Ethernet originated at a 10meg half duplex shared environment working with collisions and unknown delays in your traffic. Ethernet shared environments will allow for any device to send traffic over the network at any given time. The concern comes in when two devices send information simultaneously; when this occurs, the hubs will detect a collision in the traffic and send a message to all devices on the network to wait a random amount of time for the collision to finish before attempting to retransfer the data. Thus a shared network environment was sufficient for getting the job done but not for being able to offer higher timing accuracy needed for automation systems.
Thankfully this is a thing of the past and now 10/100 Meg and 1 Gig network speeds are available in full duplex switched environments catering for the speeds and turn around times of only milliseconds for automation to take place in a predictable, reliable and safe manner. A switched full duplex environment as opposed to shared half duplex, can be simply explained by making use of an analogy of a car travelling on a highway. Half duplex shared would be a one lane highway with traffic travelling in both directions! When the two cars collide, then both ends have to stop transmitting until the accident is cleared up. Thereafter cars can commute one at a time to their destinations. A full duplex switched environment is more like a multiple lane highway, catering for travel in both directions simultaneously.
Another comparison is a two way radio which is typically half duplex, push to speak then let go to listen, this would be half duplex fashion, where a cell phone you can talk and listen simultaneously thus improving throughput as on the full duplex switched Ethernet environment.
Switched full duplex Ethernet caters for applications from a home computer browsing the internet, running a computer game LAN, to a large Industrial Plant running their mission critical automation systems.
Now that we have some ideas on the uses of Ethernet, we can look into how the design caters for such diverse applications and their requirements.
Ethernet is built up from seven building blocks referred to as the ‘7 Layer OSI’ (Open System Interconnect) model. The model consist of the following layers:
- Physical
- Data Link
- Network
- Transport
- Session
- Presentation
- Application
The Ethernet layer is made up of the physical (medium) and data link (MAC) Layers. The physical layer would typically ensure your base connection as cable, fibre or wireless and define the connection speed. The data link layer would typically assemble the frames with the unique MAC identifier, insert the frame onto the network medium and the reverse also occurs in order to receive information from other devices. This is Ethernet. The IP Layer has already started changing from IPv4 to IPv6 for specified applications (not typically used in automation networks) and the application layer will have continuous growth of new protocols as their needs arise. The network layer maps and allocates the configured IP address to the appropriate MAC address. In other words the network layer is superimposing a logical configured address onto a physical hardware address. That device in future would be referenced to the configured IP address on the higher layers.
In order for an Ethernet network to operate correctly the following steps are traversed for the two devices communicating:
Figure 1: Areas of redundancy focus in the OSI model.
What makes it diverse is that the upper layers can be replaced with different protocols for different purposes and applications with using the same existing Ethernet infrastructure – therefore saving costs from using one infrastructure to cater for multiple purposes and applications.
Redundancy on the upper layers is not as important as redundancy on the physical Layer, since the higher layers have built in provisions for interruptions in the communication stream, however if a physical cable breaks, it is beneficial to have a redundant cable making use of a different path.
– HTTP for web browsing or IP CCTV Viewing using your choice of available streaming CODECS.
– FTP for fast file transfers without any viewing of images.
– SIP (Session Initiating Protocol) for VoIP Telephony.
– Modbus TCP and DNP3 for industrial communication: All the other industrial protocols and newly developed protocols typically come on the top of the OSI reference model at the application layer and are compatible with the standard Ethernet frame build up.
As per Figure 2 we can see how even industrial and utility requirements can be separated and catered for individually depending on their requirements.
Figure 2: Separation of utility automation and industrial automation sectors.
Once your Ethernet backbone is installed and correctly configured, it has quickly become a standard to create a VLAN (Virtual Local Area Network) on a per communication requirement level.
A VLAN is creating a subset of logical networks over a single physical infrastructure. The main reason for this logical separation is to isolate broadcast traffic to their individual logical networks thus ensuring automation traffic is not able to be affected in anyway by any other systems on the same infrastructure.
Once you have these details in check you are ready to enjoy the versatility and reliability of your industrial communication backbone that, if designed and configured correctly, will use VLAN’s to cater for the following example applications:
- SCADA.
- PLC.
- Seismic Detection.
- VoIP.
- IP CCTV.
- Fire Detection.
- Other Vendor VLANs if required.
The keys to a successful implementation and running an Ethernet-based system for the communication backbone in your plant are the following:
- Network design with IP Layout to cater for future expansion (incorrect design could drastically dampen required performance).
- Equipment for required throughput and environment:
- No fans.
- High temperature ratings
- Strong power supplies
- High MTBF (Mean Time Between Failures)
- If possible, additional fibre runs for redundant mesh topology for automated network recovery upon a fibre loss within 5 ms.
- Network management system to be pro-active upon alerts on your network – as apposed to re-active.
Substation automation has also adopted ideas from industrial Ethernet automation networks as part of their automation and protection schemes. For some of their applications, however faster times and harsher environmental conditions were a concern. The IEC61850 standard was born to cater for this adding in a 61850 protocol in the application layer. Another adaptation to the OSI layer was not to use an IP address for certain messaging known as ‘Goose’. This message is only addressed to the end devices with using the hardware address known as the MAC. PTP (Precision Time Protocol) is also used in conjunction with this to ensure time accuracy of delivery within nanoseconds.
Industrial Ethernet (switches and routers) communication in industrial automation plants
Industrial Ethernet has very rapidly become the reliable technology for automation communication within production plants, utility substations and ITS. Some of the reason for this rapid change is based on flexibility, expandability, reliability, resilience, open world standard protocols, ability to adopt legacy protocols, ease of troubleshooting and maintenance.
When most plants started with Ethernet, correct planning for future vision was not always kept in mind when considering expansion of the plant or site, future implementation of IP cameras, VoIP, fire detection and many more peripherals. Such expansion would have impact on the original design for IP structures and design layouts.
Ethernet caters for, with the use of intelligent managed switches and high performance routers, the ability to assist almost any client’s requirements regarding on the fly expansions, topology changes, breakdown support and IP layout changes with minimal downtime!
The choice of equipment to use for the communication backbone in automation systems is critical. It is critical since failure of equipment or design will impact production!
H3iSquared has a great deal of experience in topology changes and assisting client requirements.
Case study
It was requested for H3iSquared to assist a Blue Chip Client to change IP addresses of their PLCs on their automation system – imposing the least amount of downtime to the production process. H3iSquared completed development of the implementation concept with their suppliers, based in Toronto, Canada. With the completed development, the solution was depicted using an industrial grade router (RX1000), ROX (Rugged Operating System on Linux), Routing, NAT (Network Address Translation) and Masquerading. Expectations were far surpassed when the recovery after the changes and implementation to the network allowed the PLC to regain communication well within the allocated time slots of only seconds.
An industrial Ethernet automation system requires a more resilient network than a typical office network. With adding the use of managed switches into the architecture you can design your mission critical network to be redundant in case of Link/Device failure. For this instance a strong mesh topology would be required with recovery time of less than 5 ms per hop per switch. This will ensure information already streamed down the network will not be lost upon a link failure on the data streams path. The IEEE has spanning tree and rapid spanning tree protocols for this type of mesh redundancy feature, but recovery times prove to be too slow since recovery times are in seconds as opposed to milliseconds. With the correct product selection, H3iSquared was able to offer this MESH type infrastructure and offer recovery time within 5 ms per switch which surpassed the former STP and RSTP with recovery times and in terms of the size of the network that can be catered for while still recovering in time from a failure that will not effect production or PLC/SCADA communication. The eRSTP is also backward compatible to STP/RSTP for systems that require more growth without the burden of cost by replacing their existing hardware.
Once the network has a strong topology layout design and is configured and running optimally, the next step is to introduce a network management system.
A network management station is a good example of features which should be of benefit to any size infrastructure. A network management station is in aid of being proactive rather than reactive by means of depicting the current status of the network at screen login with regards to device uptime/ downtime, service availability, device management configuration changes (with options to roll back firmware options as well as the older configurations). A network management solution will also prove invaluable to trouble shooting a network when problems occur.
Conclusion
As time moves forward and technology progresses, the standard of Ethernet for communication systems and the flexibility it offers, will cause it to continue to remain a standard for many years to come with many additions and enhancements for user specific applications worldwide.
Abbreviations
CCTV – Closed Circuit Television
CODECS – CODer and DECoderS for media streaming
eRSTP – Enhanced Rapid Spanning Tree Protocol
RSTP/STP – Rapid/Spanning Tree Protocol
FTP – File Transfer Protocol
HTTP –Hyper Text Transfer Protocol
IP – Internet Protocol
ITS – Intelligent Transport Systems
NAT – Network Address Translation
OSI – Open System Interconnect
PLC – Programmable Logic Controller >>>>>>>>>>CHECK : http://en.wikipedia.org/wiki/Programmable_logic_controller
PTP – Precision Time Protocol
ROS – Rugged Operating System
SIP – Session Initiating Protocol
TCP – Transmission Control Protocol
VLAN – Virtual Local Area Network