Cloudera Hadoop expertise, Linux system admin, Python programming
Installing and configuring Hadoop components using Cloudera Manager.
• Responsible for certifying and benchmarking tools and services on existing clusters before on boarding the application teams.
• Work with architecture and engineering teams to propose new tools requires for business.
• Working with the environment management teams to setting up Hadoop users, authorizing and setting up Kerberos.
• Performance tuning for Hadoop Clusters and MR/YARN routines.
• Monitoring the Hadoop clusters through Cloudera Manager and Navigator.
• File System management for local and HDFS including space allocation. Upgrading the clusters to the latest version of CDH components.
• Troubleshooting Hadoop services and tools.
• Deploying patches and fixes to the Hadoop clusters and restarting the services.
• Work with the Hadoop distribution vendors to resolve/coordinate on any major issues.
• Monitoring ETL jobs, manage and review logs.
• Configuring NFS, Apache services in Linux Servers.
• Implementing disk quotas, security standards on Hadoop servers.
• Worked on User administration adding them on net groups and authentication with LDAP.
• Having rotational on call giving 24*7 support and responsible for setting up on calls for rest of the team.
• Installing operating systems, patches, hardware, vendor software packages, system customization and Documentation.
• Creating Hive DBs on both encrypted and non-encrypted zones for application teams.
• Developed an automated map reduce activity table for query optimization and performance tuning. Backup and Recovery plan preparation and execution.
• Developed and implemented BASH Shell Scripts and Cron Jobs for LINUX/UNIX system management activities like disk quota allocation, monitoring and clean up, test lane set up and configuration of client tools like nzclient on LINUX machines. Created and implemented ssh-keys to allow password less file transfer between LINUX file systems.
• Implemented NFS mount points on a LINUX client