INSTALLATION DOCUMENTS BY RAVI

Sunday, October 15, 2017

Step by step Installing Apache Oozie in Hadoop single node

Prerequisites:


1. Java jdk 1.6+





2. Maven 3.3.9








3. Hadoop 2.x







Installing Oozie 4.3.0
















Setting the environment variables:




















Save and close the file

Running Oozie Build
Modify the pom.xml under /u01/hadoop/oozie-4.3.0 as below
















Now run the below command























Oozie server setup
Add or modify hadoop core-site.xml under /u01/hadoop/hadoop-2.7.2/etc/hadoop as below












Create a libext folder under the oozie directory





Move downloaded ext-2.2.zip to libext folder





Copy Hadoop libraries into libext folder of oozie

























Preparing war file


















Creating Oozie Sharelib

Copy the hadoop core-site.xml file properties to core-site.xml file in /u01/hadoop/oozie-4.3.0/distro/target/oozie-4.3.0-distro/oozie-4.3.0/conf/hadoop-conf 






















Copy mapred-site.xml and yarn-site.xml files to /u01/hadoop/oozie-4.3.0/distro/target/oozie-4.3.0-distro/oozie-4.3.0/conf/hadoop-conf








Add or modify oozie-site.xml file under /u01/hadoop/oozie-4.3.0/distro/target/oozie-4.3.0-distro/oozie-4.3.0/conf as below























Now run the below command

































Creating Oozie database















Setting environment variables for Oozie




















Starting Oozie daemon
Run the below command to start Oozie daemon



















Run the below command to run as a process



Setting up client node for Oozie







Starting Oozie client node




Oozie web console url

















Checking the status of the Oozie process














Checking the what sharelib is being used by oozie while the daemon is running









Running the example Oozie jobs and testing the installation













We see different example folders such as pig,hive,mapreduce etc..,
Taking mapreduce to explain below steps. 
In the job.properties files of each folder, change the name node port and job tracker port as below



















save and close the file
Copying the examples folder to hdfs







Run the below command to check the status of the job






















To check the log run the below command






















33 comments:

  1. For an individual who is terrible that both of these, destiny chose my fate yet a tad of support at my end, getting the course I needed. data science course in pune

    ReplyDelete
    Replies
    1. Google sees high potential in reinforcement learning techniques using deep neural networks for qubit control optimization. Their abilities to harness non-local regularities of noisy control trajectories and to facilitate transfer learning between tasks have inspired researchers to adopt control methods built on deep reinforcement learning.Skyrocket in your Artificial Intelligence career with the twin engines of Python and R programmings. Join our Machine Learning using Python and R program and learn to script winning Machine Learning algorithms in Python and R. Use Python and R to enable regression analysis and to build predictive models. Orient yourself with Black Box techniques like Neural Networks and SVM.
      machine learning courses

      Delete
  2. Great post i must say and thanks for the information.
    digital marketing course

    ReplyDelete
  3. Attend The Machine Learning course in Bangalore From ExcelR. Practical Machine Learning course in Bangalore Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Machine Learning course in Bangalore.
    Machine Learning course in Bangalore

    ReplyDelete
  4. Well, the most on top staying topic is Data Science. Data science is one of the most promising technique in the growing world. I would like to add Data science training to the preference list. Out of all,Data science course in Mumbai
    is making a huge difference all across the country. Thank you so much for showing your work and thank you so much for this wonderful article.

    ReplyDelete
  5. Such a very useful article. I have learn some new information.thanks for sharing.
    data scientist course in mumbai

    ReplyDelete

  6. Excelr is providing emerging & trending technology training, such as for data science, Machine learning, Artificial Intelligence, AWS, Tableau, Digital Marketing. Excelr is standing as a leader in providing quality training on top demanding technologies in 2019. Excelr`s versatile training is making a huge difference all across the globe. Enable ?business analytics? skills in you, and the trainers who were delivering training on these are industry stalwarts. Get certification on "
    best data science courses in hyderabad"
    and get trained with Excelr.

    ReplyDelete
  7. This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck.
    Data science course in Mumbai

    ReplyDelete
  8. Nice Blog...Very interesting to read this article. I have learn some new information.thanks for sharing.
    ExcelR Mumbai

    ReplyDelete
  9. Very nice blog here and thanks for post it.. Keep blogging...
    ExcelR data science training

    ReplyDelete
  10. Attend The PMP Certification in Abu Dhabi From ExcelR. Practical PMP Certification in Abu Dhabi Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The PMP Certification in Abu Dhabi.
    ExcelR PMP Certification in Abu Dhabi

    ReplyDelete
  11. This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck.
    ExcelR data analytics courses

    ReplyDelete
  12. I have to search sites with relevant information on given topic and provide them to teacher our opinion and the article.
    data science course in mumbai

    ReplyDelete
  13. Well, the most on top staying topic is Data Analytics. Data Analytics is one of the most promising technique in the growing world. I would like to add Data Analytics training to the preference list.
    Out of all, data analytics course in hyderabad is making a huge difference all across the country. Thank you so much for showing your work and thank you so much for this wonderful article.

    ReplyDelete
  14. Nice Post...I have learn some new information.thanks for sharing.
    data analytics course hyderabad

    ReplyDelete
  15. This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck.
    data analytics course mumbai

    ReplyDelete
  16. Great post i must say and thanks for the information. Education is definitely a sticky subject. However, is still among the leading topics of our time. I appreciate your post and look forward to more.
    data analytics course mumbai

    ReplyDelete
  17. Great post i must say and thanks for the information. Education is definitely a sticky subject. However, is still among the leading topics of our time. I appreciate your post and look forward to more.
    data analytics course mumbai

    ReplyDelete
  18. Awesome blog. I enjoyed reading your articles. This is truly a great read for me. I have bookmarked it and I am looking forward to reading new articles. Keep up the good work!. data science courses

    ReplyDelete
  19. Actually I read it yesterday but I had some thoughts about it and today I wanted to read it again because it is very well written.
    .Please check ExcelR Data Science Course in Pune

    ReplyDelete
  20. Attend The Machine Learning courses in Bangalore From ExcelR. Practical Machine Learning courses in Bangalore Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Machine Learning courses in Bangalore.
    ExcelR Machine Learning courses in Bangalore

    ReplyDelete
  21. This is a wonderful article, Given so much info in it, Thanks for sharing. CodeGnan offers courses in new technologies and makes sure students understand the flow of work from each and every perspective in a Real-Time environmen python training in vijayawada. , data scince training in vijayawada . , java training in vijayawada. ,

    ReplyDelete
  22. keep up the good work. this is an Ossam post. This is to helpful, i have read here all post. i am impressed. thank you. this is our data analytics courses mumbai
    data analytics course mumbai | https://www.excelr.com/data-analytics-certification-training-course-in-mumbai

    ReplyDelete
  23. It's really nice and meanful. it's really cool blog. Linking is very useful thing.you have really helped lots of people who visit blog and provide them usefull information.

    data science course
    360DigiTMG

    ReplyDelete
  24. You might comment on the order system of the blog. You should chat it's splendid. Your blog audit would swell up your visitors. I was very pleased to find this site.I wanted to thank you for this great read!!

    data analytics courses

    data science interview questions

    business analytics courses

    data science course in mumbai

    ReplyDelete
  25. The information provided on the site is informative. Looking forward for more such blogs. Thanks for sharing .
    Artificial Inteligence course in Delhi
    AI Course in Delhi

    ReplyDelete
  26. You might comment on the order system of the blog. You should chat it's splendid. Your blog audit would swell up your visitors. I was very pleased to find this site.I wanted to thank you for this great read!!
    Know more Data Scientist Course in Pune

    ReplyDelete
  27. Your work is very good and I appreciate you and hopping for some more informative posts
    Know more Data

    ReplyDelete

  28. Excellent! I love to post a comment that "The content of your post is awesome" Great work!

    top data analytics courses in mumbai

    ReplyDelete

Loading xml file data to oracle table using python

Sample xml file (test.xml): <?xml version="1.0"?> <data>     <customer name="Ravi" >     ...