Getting Started: Data Engineer Role

Jump to Section

What Does This Article Cover?

Intelligence Hub is used to build inherently unique data exchange solutions between one or more systems. These data exchange solutions may be quickly configured with no- or low-code. The purpose of this article is to describe the individuals who build data exchange solutions using Intelligence Hub. This article includes the following content.

  • What is the Data Engineer Role?
  • Attributes of an ideal Data Engineer
  • Common Data Engineer Role tasks

What is the Data Engineer Role?

The Data Engineer Role may be fulfilled by one or more individuals who configure data flow solution(s) using the Intelligence Hub. The role may be a corporate or site resource and the individual(s) for the role may vary per use case and or site. Individual aptitude is often more important than specific job function, and we frequently find that the Data Engineer Role is someone from a corporate or site Information Technology team, a Controls Engineer, or someone from a Data Science or Analytics team.

Attributes of an ideal Data Engineer:

The individual(s) in the Data Engineer Role should have a deep understanding of the respective use case as well as knowledge and access to the source and target system. It is beneficial if the individual has general technical aptitude and experience with network security, OT systems and data, and JavaScript. Optimally the resource(s) for the Data Engineer role possess the following skills.

  • A comprehensive understanding of the use case(s)
  • Knowledge of the source system(s), these are often OT systems
  • Knowledge of target systems, sometimes these are cloud services
  • Knowledge of industrial protocols like OPC UA and MQTT
  • A basic understanding of networks and network security
  • Ability to write snippets of JavaScript code

Common Data Engineer Role Tasks:

We find that the Data Engineer role often completes the following tasks.

  • Learning how to use the Intelligence Hub via the Trial Accelerator, YouTube videos, User Guide, and Knowledge Base
  • Installing Intelligence Hub or communicating installation instructions and requirements to an IT team
  • Communicating network security requirements to the appropriate internal team
  • Designing the data models per use case considering available data in source systems and the capabilities of the target systems
  • Configuring Connection Inputs in the Intelligence Hub to obtain information
  • Configuring Models, Flows, and Pipelines in the Intelligence Hub
  • Configuring Connection Outputs in the Intelligence Hub to provide information to target systems
  • Tuning the performance of the Intelligence Hub solution
  • Configuring Intelligence Hub Roles, Claims, and Users