Click here to Skip to main content
15,881,938 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
I've been able to use Sqoop & Flume import commands perfectly fine on Ubuntu terminal. But right now, I'm trying to do so on Jupyter notebook.

1) How can I import from MySQL to HDFS using Sqoop command on Jupyter notebook?
2) what is Flume import python code?

What I have tried:

In terminal:
FLUME
Python
Flume-ng agent -n TwitterAgent --conf-file /home/amel/flumeconf --conf /home/amel/flumeconf


SQOOP
Python
Sqoop list-databases --connect jdbc:mysql://localhost:3306/testDb --username Amel --password Amel@-1998;


JUPYTER
Python
import subprocess
import logging
from pysqoop.SqoopImport import Sqoop 

sqoop = Sqoop(fs='hdfs://localhost:50070',
table='mock3', target_dir='hdfs://user/amel',connect='jdbc:mysql://localhost:3306/testDb',
username='Amel', password='Amel@-1998', num_mappers=2,
bindir='/usr/local/hadoop/sqoop/bin')

sqoop.perform_import()


this is the error i get

Python
sqoop import None  -fs hdfs://localhost:50070 --table mock3 --target-dir hdfs://user/amel --connect jdbc:mysql://localhost:3306/testDb --username Amel --password Amel@-1998 --num-mappers 2 --bindir /usr/local/hadoop/sqoop/bin --help 
name 'run' is not defined
90
Posted
Updated 3-Aug-21 22:26pm
v3
Comments
Richard MacCutchan 4-Aug-21 4:29am    
This looks like an issue with sqoop; see Sqoop -[^].

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900