Database admins and other professionals who are planning to switch to Hadoop world as Hadoop Administrator should be aware of what is expected from them. Below are just some of the basic requirements that this job demand. (This is not Hadoop Developer related activites but the Admin part of Hadoop)
- Responsible for implementation and administration of Hadoop infrastructure.
- Keep expanding Hadoop existing environments by deploying new hardware/software as required.
- Setup new Hadoop users. Hadoop users may need access to HDFS, Hive, Pig and MapReduce, so Hadoop Admin needs to know these technologies.
- Cluster maintenance tasks like creation and removal of nodes. Hadoop admin may need to learn some of the third party tools to handle this responsibility.
- Design, implementation and assignment of security schemas.
- Performance tuning of Hadoop clusters and Hadoop MapReduce routines.
- Monitor Hadoop cluster connectivity and security. Security is as important as it was for the DBAs while administering database/application.
- Supporting and maintening of HDFS. It is where all the unstructured data is kept.
- Hadoop ecosystem support and maintenance (HDFS, Zookeeper, YARN, Spark, Storm, Solr, HBase, Pig, Hive) and if your organization is using other big data technologies such as ElasticSearch, NoSQL DBs, etc. then you may be involved in that too.
- Applying Hadoop updates, patches, version upgrades when required.
- Team up with the infrastructure, network, database, application and business intelligence teams to ensure an efficient Hadoop system that meets business demands.
- Reviewing Hadoop log files for troubleshooting.
I’m an experienced Cloud/Oracle Applications/DBA Architect with more than 15 years of full-time DBA/Architect experience. I have gained wide knowledge on Oracle and Non-Oracle software stack running on-prem and on Cloud and have worked on several big projects for multi-national companies. I enjoy working with leading-edge technology and have a passion for Cloud architecture, automation, database performance, and stability. Thankfully my work allows me time for researching new technologies (and to write about them).
Latest posts by Brijesh Gogia (see all)
- Oracle Multitenant DB 4 : Parameters/SGA/PGA management in CDB-PDB - July 18, 2020
- Details of running concurrent requests in Oracle EBS - May 26, 2020
- Oracle Multitenant DB 3 : Data Dictionary Architecture in CDB-PDB - March 20, 2020