Transforming timestamps into matlab fft usable form?

Question

0.00/5 (No votes)

See more:

Hi all! i have the following timestamps of detection time below and tens of thousands more not not posted.
I am dying to know how to transform these numbers into a matlab FFT function usable form so that i can find the period of the signal. Pls help out this beginner here!

VB

82473035994223.000000
82473036321275.000000
82473036758714.000000
82473037152621.000000
82473037463611.000000
82473037735504.000000
82473038019083.000000
82473038281295.000000
82473038636703.000000
82473039013925.000000
82473039348376.000000
82473039596255.000000

What i have thought of is this, by giving the timestamps a value of 1, and trying to fill out the in betweens times with values of 0. But i do not know how to implement the action of filling out the in betweens with 0 and how to put it into the fft. pls advise!

VB

82473035994223.000000    1
82473036321275.000000    1
82473036758714.000000    1
82473037152621.000000    1
82473037463611.000000    1
82473037735504.000000    1

Posted 10-Feb-14 1:37am

Member 10512547

Add a Solution

3 solutions

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Matt T Heffron · Answer 1 · 2014-02-10T08:58:00

Solution 1

I'm not sure it is practical, but a way to generate a vector where there's a 1 at the data positions and 0 everywhere else would be something like:

rawTimes = [82473035994223.000000
82473036321275.000000
82473036758714.000000
82473037152621.000000
82473037463611.000000
82473037735504.000000
82473038019083.000000
82473038281295.000000
82473038636703.000000
82473039013925.000000
82473039348376.000000
82473039596255.000000
# etc.];
reducedTimes = fix(rawTimes .- rawTimes(1));  # shift the absolute times to be INTEGER offsets
eventData = zeros(1, 1+reducedTimes(end));
eventData(1+reducedTimes) = 1;
# now you can fft(eventData);

This eventData vector will likely be HUGE.

Posted 10-Feb-14 8:58am

Matt T Heffron

Comments

Member 10512547 10-Feb-14 15:20pm

you were right,i tried this and i got out of memory.

Matt T Heffron 10-Feb-14 15:40pm

Can you reduce the SPAN of the data (i.e., size of eventData) by reducing the precision of the data?
E.g., if the time stamps are in seconds but you only need the calculation to hours, then divide rawTimes by 3600 before calculating reducedTimes (and probably changing fix() to round() ). That reduces the size of the eventData vector by a factor of 3600.

Albert Holguin 10-Feb-14 20:25pm

I gave him some suggestions... basically has to normalize the data then run in blocks...

Matt T Heffron 10-Feb-14 15:45pm

You probably ought to see if you can use Matlab sparse matrices, and if they're compatible with fft().
Something like:
eventData = sparse(1+reducedTimes, 1, 1);
This could work with fft(). Assuming it doesn't forcibly expand the vector internally...

Albert Holguin · Answer 2 · 2014-02-10T14:17:00

Solution 2

Just put it in a vector with seconds (transform whatever that is to seconds, make the first sample be zero and increase from there, in another words, normalize your values) and break up the data into blocks and you can do an fft() on it. The block will have a spectral resolution equal to fs/N, where fs is the sampling frequency and N is the block size (the fft routine will pick the fft size based on the block size that you pass to it, using a 2^N number).

Posted 10-Feb-14 14:17pm

Albert Holguin

Comments

Member 10512547 10-Feb-14 22:18pm

i have normalised it as u said, but what do you mean by breaking up into blocks? do i just fft(normaliseddata)?

Albert Holguin 10-Feb-14 22:30pm

How many data samples do you have? ...if it's a huge number, then you can do a faster FFT by breaking it up into smaller chunks, and you'll still get the information you need. For example, you can break it up into chunks of 8K data points and feed that into the FFT. The FFT will essentially give you a spectral window into that chunk of time, you can move your window over as needed.

Member 10512547 10-Feb-14 22:32pm

i have about more than one million points.

Albert Holguin 10-Feb-14 22:36pm

yeah, well break that up into some number, preferably some amount that is a power of two (2^N), since that's what the fft routine will use (if you don't feed it exactly that, it zero pads).

Member 10512547 10-Feb-14 22:43pm

is this correct then?

using 8192 points

X = rawTimes - rawTimes(1);/* normalising*/
X = X*(10e^-12)*81; /* since 1 time stamp unit is 81ps */

plot(fft(X,8192));

Albert Holguin 10-Feb-14 22:49pm

Not even close...lol... break up the data into smaller segments, SmallerSegment = X[1:8192]; would be your first segment. Also, you can't just plot your fft output, you have positive and negative frequencies, if you only want positive (real valued signal), plot from 1:fft_length/2. If you're also interested in the negative frequencies, then you have to call fft_shift() before plotting.

Member 10512547 10-Feb-14 22:56pm

ok im lost there, pls tell me what to do

Albert Holguin 10-Feb-14 23:20pm

plot half of your fft output, and feed in chunks... each plot will represent the spectral display of that chunk, you can plot them to different graphs and you'll see the spectral changes in time.

Albert Holguin 10-Feb-14 22:50pm

...and what do you mean by 81ps?

Member 10512547 10-Feb-14 23:05pm

1 timestamp units is equal to 81picoseconds

Albert Holguin 10-Feb-14 23:23pm

by the way... if you're just doing an fft on time stamps, you'll probably not get anything eventful, the fft should be on the data that the time stamping is associated with.

Albert Holguin 10-Feb-14 23:34pm

if you're wondering what happens when you take an FFT of time stamps... well, you'll get nothing, because the DFT is essentially a window into the spectral domain of a signal when sampled at some interval of time. If your timestamps ARE your data, and they occur at a regular interval, then there's really nothing there to show.

Matt T Heffron · Answer 3 · 2014-02-10T16:18:00

Solution 3

Thinking about it more, I'm not sure you really need to use an FFT.
If you just want the best approximation to the period of the signal, then maybe you should use Linear Regression.
Use something like:

rawTimes = [82473035994223.000000
82473036321275.000000
82473036758714.000000
82473037152621.000000
82473037463611.000000
82473037735504.000000
82473038019083.000000
82473038281295.000000
82473038636703.000000
82473039013925.000000
82473039348376.000000
82473039596255.000000
# etc.];
X = rawTimes .- rawTimes(1);  # unlike Solution 1, these no longer need to be integer
Y = 1:(size(X)(1));
fit = polyfit(X, Y', 1);
period = 1./fit(1);

period is the best fit, in timestamp units, of the interval between "events"
With these 12 points the period is about 325441.8

Posted 10-Feb-14 16:18pm

Matt T Heffron

Updated 10-Feb-14 16:22pm

v3

Comments

Member 10512547 10-Feb-14 22:21pm

i got an error
>> Y = 1:(size(X)(1));
Error: ()-indexing must appear last in an index expression.

what does it mean?

Matt T Heffron 11-Feb-14 12:03pm

I don't know what the problem is here, I used this EXACT statement when I tried this.

Matt T Heffron 11-Feb-14 12:30pm

Perhaps:
[XL, ignore] = size(X);
Y = 1:XL;

Member 10512547 10-Feb-14 22:26pm

Also,i forgot to mention that within the detection timestamps i obtained, some might be darkcounts, meaning a false detection so this method might not really work.

Matt T Heffron 11-Feb-14 12:01pm

If you know which are which, use only the ones that are positive events.

Matt T Heffron 11-Feb-14 12:28pm

If you better describe the problem and available data characteristics, we can give better advice.

Albert Holguin 10-Feb-14 22:33pm

fft's don't require integers... they typically use single or double precision floating types.

Matt T Heffron 11-Feb-14 12:02pm

Yes, but in my Solution 1, the values had to be integer because they were being used as INDICES to set the values into the vector.

Albert Holguin 11-Feb-14 13:16pm

Oh, I see... but it's still not practical to feed that many points to an fft routine

Matt T Heffron 11-Feb-14 13:19pm

Which is why this solution proposes Linear Regression!
The slope of the fit line is the frequency of occurrence, 1/slope is the period.

Albert Holguin 11-Feb-14 14:18pm

I was referring to your other solution. Settle down.