Creating bins of 3D points (large dataset) with Python, taking long time to load

See more:

Python

3D

statistics

data-science

I am trying to make a 3D plot of a galaxy catalog and have a large amount of x,y,z coordinates and data value (w4) stored in seperate hdf5 files.

Since the data content is huge, I have tried binning them.

The output is however taking forever to load and I have tried various binning techniques (binned_statistics_dd, histogramdd etc) but nothing has worked.


Any help will be appreciated since I have been trying this for weeks now.

What I have tried:

The code is as:

    gaspos = np.array(gas['Coordinates'])*ckpc/h  ##coordinates
    x = gaspos[:,0]
    y = gaspos[:,1]
    z = gaspos[:,2]
    
    w4 = gas['MagneticField']*(h/a**2)*np.sqrt(1e10*Msun/kpc)*1e5/kpc
    w4 *= w4/(8*np.pi)
    w4 = (np.dot(w4,np.ones((3,1))).T)[0]  ## 1 dimension data
    
    hist, binedges = np.histogramdd(gaspos, normed=False)
    hist, binedges = np.histogramdd(w4, normed=False)
    
    fig = plt.figure(figsize = (16, 9))
    ax1 = plt.axes(projection ="3d")

    ax1.scatter3D(x,y,z,c = w4)
    plt.show()