Data Engineer
Responsibilities include but are not limited to:
- Design and develop high quality, scalable software modules for next generation analytics solution suite
- Prototype high impact innovations, catering to changing business needs, by leveraging new technologies (AWS – Cloud and Big Data)
- Possesses functional knowledge and skills reflective of a competent practitioner with the ability to deliver on work of highest technical complexity
- Consults with functional management in the analysis of short and long-range business requirements and recommends innovations which anticipate the future impact of changing business needs
- Works closely with client management to identify and specify the complex business requirements and processes for diverse development platforms, computing environments (e.g., Cloud, host based, distributed systems, client server), software, hardware, technologies and tools
- Provides highly technical consulting and leadership in identifying and implementing new uses of information technologies that assist the functional business units in meeting their strategic objectives
- Reviews the disaster recovery plan to ensure that new systems promote uninterrupted systems operation
- Coordinate activities with cross-functional IT unit stakeholders (e.g., database, operations, telecommunications, technical support, etc.)
- Researches and evaluates alternative solutions and recommends the most efficient and cost-effective solution for the systems design
- Formulates logical statements of business problems and devises, tests and implements efficient, cost-effective application program solutions (e.g., codes and/or reuses existing code through the use of program development software alternatives and/or integrates purchased solutions)
- Prepares charts, tables and diagrams to assist in analyzing problems, utilizing various business, scientific, engineering and mathematical techniques. Analyzes existing system and programming logic to provide more efficient machine operations or to identify difficulties, and revises the logic and procedures involved as necessary
- Primary technical point of contact for interaction with other cross-functional units and teams.
- Assign personnel to the various projects or application components, and direct their activities
- Review, evaluate and prepare periodic activity and progress reports for the team
Qualifications
- Bachelor’s degree in Computer Science or a related discipline and 5 or more years’ experience in IT systems analysis and application program development, or an equivalent combination of education and work experience
- 1+ years ETL or Data Integration experience
- 2+ years of Big Data experience
- Cloud (AWS) Technologies experience preferred
- AWS or Big Data Technologies certification preferred
- Knowledge of Oracle, SQL, Python, Pyspark, Unix Or Linux Shell scripting, Autosys scheduling tool, version control Tools
- Strong data warehouse applications knowledge
- Knowledge of Data modeling, Big Data Distributed Processing (HDFS, Hive, Spark), AWS, Data Analytical languages Python preferred
- Good programming skills in Python