Click here to Skip to main content
15,885,998 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
Hi i have a DataSet of Track.class i want to merge all tracks that are within same interval of time for example 5 min .i.e any tracks start after a track that ends within 5 min before will be the same track.its look like fusion task.

my input :

+----------+---------------------------------------------+
|  trackId |  start_time    |  end_time                  |
+-----+--------------------------------------------------+
|  1       | 12:00:00       |   12:04:00                 |
+----------+---------------------------------------------+
|  2       | 12:05:00       |   12:08:00                 |  
+----------+---------------------------------------------+
|  3       | 12:20:00       |   12:22:00                 | 
+----------+---------------------------------------------+
output :(trackId : 1,2 are been merged since diff of start and end of each one is within 5 min)

+----------+----------------+----------------------------+
|  trackId |  start_time    |  end_time                  |
+-----+--------------------------------------------------+
|  1       | 12:00:00       |  12:08:00                  |
+----------+---------------------------------------------+
|  3       | 12:20:00       |   12:22:00                 |
+----------+----------------+----------------------------+
so how i can make do that ?


What I have tried:

df.groupBy(window(df.col("time"),"50 seconds").agg(first("id").as("id"))

Posted
Updated 23-Apr-18 23:52pm

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900