As a Hadoop Engineer, you will be responsible for planning, design and development of platform features/capabilities leveraging open source, Hadoop-related technologies. Candidates must be familiar with Agile development methods, understand object-oriented programming methodology, able to collaborate with other architects on component design, and experienced with the software development lifecycle.
1. Hadoop development and implementation.
2. Loading from disparate data sets.
3. Pre-processing using Hive and Pig.
4. Designing, building, installing, configuring and supporting Hadoop.
5. Translate complex functional and technical requirements into detailed design.
6. Perform analysis of vast data stores and uncover insights.
7. Maintain security and data privacy.
8. Create scalable and high-performance web services for data tracking.
9. High-speed querying.
10. Being a part of a POC effort to help build new Hadoop clusters.
11. Test prototypes and oversee handover to operational teams.
12. Propose best practices/standards.
Desired Skills and Experience
1. Knowledge in Hadoop
2. Good knowledge in back-end programming, specifically java
3. Writing high-performance, reliable and maintainable code.
4. Ability to write MapReduce jobs.
5. Good knowledge of database structures, theories, principles, and practices.
6. Ability to write Pig Latin scripts.
7. Hands on experience in HiveQL.
8. Familiarity with data loading tools like Flume, Sqoop.
9. Knowledge of workflow/schedulers like Oozie.
10. Analytical and problem solving skills, applied to Big Data domain
11. Proven understanding with Hadoop, HBase, Hive, Pig, and HBase.
12. Good aptitude in multi-threading and concurrency concepts.