I try to use Multiprocessing in a class.I use Multiprocessing.Pipe() to pass the instance o from parent process to child process.
Here I call the Multiprocessing.pipe():-
conn1, conn2 = multiprocessing.Pipe()
self.devices = bussiness.initilaztion(self.comm_object,logger)
conn1.send(self.devices)
self.callmotor1dprocess =
multiprocessing.Process(target=callallmotor1d,args=[conn2])
self.listofthread.append(self.callmotor1dprocess)
self.callsov1sprocess =
multiprocessing.Process(target=callallsov1s,args=[conn2])
self.listofthread.append(self.callsov1sprocess)
Now Executed method I should call outside of the class.
def callallmotor1d(conn):
devices = conn.recv()
while True:
bussiness.motorallprocessing(devices)
def callallsov1s(conn):
while True:
devices = conn.recv()
bussiness.allsov1processing(devices)
Now I call all Multiprocessing Initiate <pre>
<pre lang="Python">
def startprocess(self):
for item in self.listofthread:
item.start()
self.button3.config(text="started")
def stopprocess(self):
for item in self.listofthread:
item.kill()
But I got the error which is very common:-
Error message:-
can't pickle _thread.lock objects
Traceback (most recent call last):
File "C:/Users/misu01/PycharmProjects/opcua/ui_v7.py", line 226, in initilization conn1.send(devices)
File "C:\Users\misu01\AppData\Local\Programs\Python\Python37\lib\multiprocessing\connection.py", line 206, in send self._send_bytes(_ForkingPickler.dumps(obj))
File "C:\Users\misu01\AppData\Local\Programs\Python\Python37\lib\multiprocessing\reduction.py", line 51, in dumps cls(buf, protocol).dump(obj)
TypeError: can't pickle _thread.lock objects
I don't know why thread.lock object is created in my Alldevices class.I never used thread.lock in my class.
What I have tried:
To avoid this error I try to modify my Alldevices class like this:-
I use __setstate__ and __getstate__ in my Alldevices class.
As per the document:-
<a href="https://docs.python.org/3/library/pickle.html#pickling-class-instances">pickle — Python object serialization — Python 3.7.4 documentation</a>[<a href="https://docs.python.org/3/library/pickle.html#pickling-class-instances" target="_blank" title="New Window">^</a>]
Here is my modification:-
class AllDevices:
def __init__(self,comobject,logger):
self.mylock = threading.Lock()
self._comobject = comobject
dfM1D = pd.read_excel(r'D:\OPCUA\Working_VF1.xls', sheet_name='Motor1D')
dfM2D = pd.read_excel(r'D:\OPCUA\Working_VF1.xls', sheet_name='Motor2D')
dfsov1S = pd.read_excel(r'D:\OPCUA\Working_VF1.xls', sheet_name='Valve1S')
dfsov2S = pd.read_excel(r'D:\OPCUA\Working_VF1.xls', sheet_name='Valve2S')
dfanalog = pd.read_excel(r'D:\OPCUA\Working_VF1.xls', sheet_name='AnalogTx')
dfcontrolvalve = pd.read_excel(r'D:\OPCUA\Working_VF1.xls', sheet_name='ControlValves')
dfvibrofeeder = pd.read_excel(r'D:\OPCUA\Working_VF1.xls', sheet_name='VibroFeeder')
dfconveyor = pd.read_excel(r'D:\OPCUA\Working_VF1.xls', sheet_name='Conveyor')
dfdrives = pd.read_excel(r'D:\OPCUA\Working_VF1.xls', sheet_name='Drive')
self.allmotor1dobjects = callallmotor1D_V1.Cal_AllMotor1D(dfM1D, comobject, logger)
self.allmotor2dobjects = callallmotor2D_V1.Cal_AllMotor2D(dfM2D,comobject,logger)
self.allsov1sobjects = callallsov1S_V1.Cal_AllSov1S(dfsov1S,comobject,logger)
self.allsov2sobjects = callallsov2S_V1.Cal_AllSov2S(dfsov2S, comobject,logger)
self.allanalogobjects = calallanalog_V1.Cal_AllAnalogInputs(dfanalog,comobject,logger)
self.allcontrolvalveobjects = calallcontrolvalves_V1.Cal_AllControlValve(dfcontrolvalve,comobject,logger)
self.allvibrofeederobjects = callallvibrofeeder_V1.Cal_AllVibroFeeder(dfvibrofeeder,comobject,logger)
self.allconveyorobjects = callallconveyor_V1.Cal_AllConveyor1D(dfconveyor,comobject,logger)
self.allabbdrivesobjects = calallABPdrives_V1.Cal_AllDrives(dfdrives, comobject,logger)
def __getstate__(self):
state = vars(self).copy()
del state['mylock']
return state
def __setstate__(self, state):
vars(self).update(state)
self.mylock = threading.Lock()
@property
def allmotor1d(self):
return self.allmotor1dobjects
@property
def allsov1s(self):
return self.allsov1sobjects
But still I got an error.
Is this my implementation is correct?
Here self.allmotor1dobjects, self.allsov1sobjects etc are also class instances.I suspect that as if I called instances of classes in my alldevice class that reason it is not working.
How I can remove this error.
I am not sure multiprocessing.pipe() is correct way to send instance of class to main process to child process
Also I try to know how thread.lock() object is created in my alldevice class.
any help in this regard will be highly appreciated
No improvement in error:-
2019-09-06 10:54:17,835: can't pickle _thread.lock objects
Traceback (most recent call last):
File "C:\Users\misu01\Desktop\SMS_SIMULATION\ui_v7.py", line 220, in initilization
conn1.send(devices)
File "C:\Users\misu01\AppData\Local\Programs\Python\Python37\lib\multiprocessing\connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "C:\Users\misu01\AppData\Local\Programs\Python\Python37\lib\multiprocessing\reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)