Designing and developing solutions on large scale customer BigData solutions
Data ingestion and transformation of structured and unstructured data sets to drive insights and value for customer. Designing and developing automation code for deploying and managing Hadoop solutions. Managing, upgrading and troubleshooting Hadoop clusters. Install and configure software. Develop project deliverable documentation.
- Experience designing, deploying and managing production Hadoop clusters (Apache Foundation, Cloudera, MapR, Hortonworks, IBM Big Insights) - Experience with Linux administration - Experience with shell scripting - Experience developing infrastructure automation code (e.g. Terraform and Ansible) - Solid programming skills in Python or similar scripting languages - Hardware and Software Monitoring including Network performance - Experience with design, management, implementation of Backup, Disaster Recovery and/or High Availability solutions - Strong analytical skills and creative problem solver - Strong team player capable of working in a demanding start-up environment - advanced English Preferred Knowledge, Skills and Abilities: - Hadoop management tools (CDH CM and Ambari) - Securing Hadoop clusters - Cloud platforms (AWS, Azure, GCP) - Experience with Hadoop data processing (in particular Hive and Spark) - Experience developing processing pipelines for Hadoop solutions - Experience with setting up data ingestion for Hadoop (e.g. NiFi, Sqoop, Kafka) - Experience administrating relational and NoSQL databases - Performance and capacity management
Major BigData solutions provider