Database admins and other professionals who are planning to switch to Hadoop world as Hadoop Administrator should be aware of what is expected from them. Below are just some of the basic requirements that this job demand. (This is not Hadoop Developer related activites but the Admin part of Hadoop)
- Responsible for implementation and administration of Hadoop infrastructure.
- Keep expanding Hadoop existing environments by deploying new hardware/software as required.
- Setup new Hadoop users. Hadoop users may need access to HDFS, Hive, Pig and MapReduce, so Hadoop Admin needs to know these technologies.
- Cluster maintenance tasks like creation and removal of nodes. Hadoop admin may need to learn some of the third party tools to handle this responsibility.
- Design, implementation and assignment of security schemas.
- Performance tuning of Hadoop clusters and Hadoop MapReduce routines.
- Monitor Hadoop cluster connectivity and security. Security is as important as it was for the DBAs while administering database/application.
- Supporting and maintening of HDFS. It is where all the unstructured data is kept.
- Hadoop ecosystem support and maintenance (HDFS, Zookeeper, YARN, Spark, Storm, Solr, HBase, Pig, Hive) and if your organization is using other big data technologies such as ElasticSearch, NoSQL DBs, etc. then you may be involved in that too.
- Applying Hadoop updates, patches, version upgrades when required.
- Team up with the infrastructure, network, database, application and business intelligence teams to ensure an efficient Hadoop system that meets business demands.
- Reviewing Hadoop log files for troubleshooting.
[Post Views: 1384]
You can connect with me on LinkedIn.
Latest posts by Brijesh Gogia (see all)
- Performance Tuning Basics 16 : Using SQL Tuning Health-Check Script (SQLHC) - April 1, 2018
- Physical and Logical Block Corruption in Oracle Database - March 10, 2018
- Performance Tuning Basics 15 : AWR Report Analysis - February 6, 2018