Click here to Skip to main content
15,892,253 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
I have a .CSV file containing weather Data for the whole year, I'd like to calculate the values needed for every day and print it to another .CSV file .
The .csv file looks like this :

stationid	datestamp	wgust	wspeedavg	wdir	wsector	alarmamber	alarmred			
FX32	31-12-19 23:52	20	0	155				0	0	0
FX32	31-12-19 23:50	18.99	0	159				0	0	0
FX32	31-12-19 23:48	18.01	0	154				0	0	0
FX32	31-12-19 23:42	18.99	0	152				0	0	0
FX32	31-12-19 23:40	18.99	0	151				0	0	0
FX32	30-12-19 23:30	18.99	0	144				0	0	0
FX32	30-12-19 23:28	18.99	0	147				0	0	0
FX32	30-12-19 23:22	20	0	135				0	0	0


What I have tried:

I tried this code so far , it can read the file and calculate total AVG/Min/Max. Is there a way i can specify to do this day by day ?

Java
package sortdata;


import com.opencsv.bean.CsvToBean;
import com.opencsv.bean.CsvToBeanBuilder;
import com.opencsv.bean.HeaderColumnNameMappingStrategy;

import com.opencsv.bean.StatefulBeanToCsv;
import com.opencsv.bean.StatefulBeanToCsvBuilder;

import com.opencsv.exceptions.CsvDataTypeMismatchException;
import com.opencsv.exceptions.CsvRequiredFieldEmptyException;

import java.io.BufferedReader;

import java.io.FileWriter;
import java.io.IOException;


import java.io.Writer;

import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;

import java.util.ArrayList;




import java.util.DoubleSummaryStatistics;
import java.util.IntSummaryStatistics;
import java.util.List;


import java.util.LongSummaryStatistics;
import java.util.Scanner;

import javax.xml.crypto.Data;

import org.apache.commons.collections.iterators.ObjectArrayListIterator;


public class Main {


    /**
     * @param args
     *
     */
    @SuppressWarnings({ "rawtypes", "unchecked", "oracle.jdeveloper.java.nested-assignment" })
    public static void main(String[] args) throws IOException, CsvDataTypeMismatchException,
                                                  CsvRequiredFieldEmptyException {
        

        String fileName = "C:\\JDeveloper\\wdata.csv";
        Path myPath = Paths.get(fileName);
        
        ArrayList<String>  d = new ArrayList<String>();
          convertDegreeToCardinalDirection direction = new convertDegreeToCardinalDirection();

        try (BufferedReader br = Files.newBufferedReader(myPath, StandardCharsets.UTF_8)) {    
           


            

            HeaderColumnNameMappingStrategy<Read> strategy = new HeaderColumnNameMappingStrategy<>();
            strategy.setType(Read.class);
            


            CsvToBean toBean = new CsvToBeanBuilder(br).withType(Read.class)
                                                       .withMappingStrategy(strategy)
                                                       .withIgnoreLeadingWhiteSpace(true)
                                                       .build();


            List<Read> Data = toBean.parse();                                                  
                                              
          //Calc for AVG MIN MAX WSpead

            final DoubleSummaryStatistics Wind = Data.stream()
                                                     .mapToDouble(Read::getWgust)
                                                     .summaryStatistics();

            final double averageWind = Wind.getAverage();
            final double Windmin = Wind.getMin();
            final double Windmax = Wind.getMax();
            final int Windtotal = (int) Wind.getCount();

            //calc for AVG MIN MAX Wdir

            final IntSummaryStatistics WindDIR = Data.stream()
                                                     .mapToInt(Read::getWdir)
                                                     .summaryStatistics();

            final double averageWdir = WindDIR.getAverage();
            final int maxWdir = WindDIR.getMax();
            final int minWdir = WindDIR.getMin();
           
            


            System.out.println("Total Number of Values : " + Windtotal);
            System.out.println("AverageWindSpeed: " + averageWind);
            System.out.println("Minimum WIndSpeed: " + Windmin);
            System.out.println("Maximum WIndSpeed: " + Windmax);

            System.out.println("AverageWindDir: " + averageWdir);
            System.out.println("Minimum WIndDir: " + minWdir);
            System.out.println("Maximum WIndDir: " + maxWdir);
            // System.out.println(stat);

            
                               Writer writer = new FileWriter("Datasorter1.csv");
                               StatefulBeanToCsv beanToCsv = new StatefulBeanToCsvBuilder(writer).build();
                               beanToCsv.write(Wind);
                               writer.close();
                    } //end of buffer reader
    } //end if main string  

}// end of class

Also have a Class Read wich contains the following :
Java
public class Read {
    @CsvBindByName(column = "stationid", required = true)
    private String stationid;
    @CsvBindByName(column = "datestamp", required = true)
    @CsvDate("dd-MM-yy HH:mm")
    private Date datestamp;
    @CsvBindByName(column = "wgust", required = true)
    private double wgust;
    @CsvBindByName(column = "wdir", required = true)
    private int wdir;
    
    
    private PropertyChangeSupport _propertyChangeSupport = new PropertyChangeSupport(this);


      
    

    public void setStationid(String stationid) {
        String oldStationid = this.stationid;
        this.stationid = stationid;
        _propertyChangeSupport.firePropertyChange("stationid", oldStationid, stationid);
    }

    public String getStationid() {
        return stationid;
    }

    public void setDatestamp(Date datestamp) {
        Date oldDatestamp = this.datestamp;
        this.datestamp = datestamp;
        _propertyChangeSupport.firePropertyChange("datestamp", oldDatestamp, datestamp);
    }

    public Date getDatestamp() {
        return datestamp;
    }

    public void setWgust(double wgust) {
        double oldWgust = this.wgust;
        this.wgust = wgust;
        _propertyChangeSupport.firePropertyChange("wgust", oldWgust, wgust);
    }

    public double getWgust() {
        return wgust;
    }

    public void setWdir(int wdir) {
        int oldWdir = this.wdir;
        this.wdir = wdir;
        _propertyChangeSupport.firePropertyChange("wdir", oldWdir, wdir);
    }

    public int getWdir() {
        return wdir;
    }


    public void addPropertyChangeListener(PropertyChangeListener l) {
        _propertyChangeSupport.addPropertyChangeListener(l);
    }

    public void removePropertyChangeListener(PropertyChangeListener l) {
        _propertyChangeSupport.removePropertyChangeListener(l);
    }
 
    @Override
    public String toString() {

        StringBuilder builder = new StringBuilder();
        builder.append("Station{id=")
               .append(stationid)
               .append(", Date=")
               .append(datestamp)
               .append(", Wspeed=")
               .append(wgust)
               .append(", Direction=")
               .append(wdir)
               .append("}");

        return builder.toString();
    }


I would appreciate every help (sample code if possible), i do not have lot of experience in coding, and this would be a on of job.
Posted
Updated 27-Mar-20 5:41am
Comments
Richard MacCutchan 27-Mar-20 10:53am    
You just need to add a selection test to read take only the records that have the correct date. In a similar way you could select records that have the same month.
OriginalGriff 27-Mar-20 11:05am    
I'm guessing that he's copy'n'pasted the code and has no idea how it works, much less how to modify it ... :sigh:
Szilard Csedo 27-Mar-20 11:17am    
Unfortunatelly i started java 3 days ago, and have a little bit of experience from school about c++, so i have zero experience, but so far i managed to put this together from lessons, tutorials, sample codes, even had to learn how to use API's (most of the publishers recommended OPENCSV) i have learned a lot , but there is much more, and every subject is related which makes it really difficult for me :) . I would appreciate it if you could point me to the right direction. Thank you
Maciej Los 27-Mar-20 11:27am    
The name of class is bit confusing, because [Read] is rather used to define method.
Szilard Csedo 27-Mar-20 12:05pm    
Thanks for claryfing, have renamed it to ReadW.

1 solution

As far as i can see, you are able to get summaryStatistics for entire list:
Java
List<Read> Data = toBean.parse();  
//skipped lines
inal DoubleSummaryStatistics Wind = Data.stream()
                               .mapToDouble(Read::getWgust)
                               .summaryStatistics();


All, what you need to do is to get specific portion of data (month, quarter, year) into another list then to call summaryStatistics() method for that smaller-list.
 
Share this answer
 
Comments
Szilard Csedo 27-Mar-20 13:06pm    
Please correct me if i am wrong but ,this way i would only be able to do a Monthly average, if i copy only the day and the relevant information how can i specify when will the next day start ?
Maciej Los 27-Mar-20 15:10pm    
As i mentioned, you need to define method which will get a "full-list" and return a list of data containing monthly, quarterly or yearly data.
If you would like to define a universal method then you need to pass 2 dates into it. The first - defines the beginnning of the date range, the second - defines the end of the date range. Think of it and start coding.
[Edit]
If you would like to get daily data, you need to call method to get smaller-list inside a loop. So, get the mininimum and maximum date of csv data, then to run loop.
Szilard Csedo 30-Mar-20 13:37pm    
I am getting an input String from console with the relevant "Date" with the following format ("dd-MM-yy") then convert that to Date using simpleDateFormat.
in my Method i get this Date and use it to compare with my "datestamp" column from the .csv file. But my file contains Date the follwing format ("dd-MM-yy HH:mm") and i believe this is the issue causing me problems with compares .
Hide   Copy Code
if (date.equals(val))...

Where date is from records, and val is from console input .
Can you suggest a solution how to ignore irrelevant fields ?
Maciej Los 30-Mar-20 15:05pm    
Seems, you'ra talking about comparing strings instead of dates. Convert both dates (comming from console and csv) into proper datetime data type.
string consoledate = "30-03-20";
string csvdate = "30-03-20 20:49";

System.Globalization.CultureInfo ci = System.Globalization.CultureInfo.InvariantCulture;
DateTime dt1 = DateTime.ParseExact(consoledate, "dd-MM-yy", ci);
DateTime dt2 = DateTime.ParseExact(csvdate, "dd-MM-yy HH:mm", ci);
dt2 = dt2.AddHours(-dt2.Hour).AddMinutes(-dt2.Minute).AddSeconds(-dt2.Second);

var result = dt1.Equals(dt2);
Szilard Csedo 31-Mar-20 4:59am    
Sorry to be a pain but , how can I implement this in Java ? DateTime is a java package but i could not find one for "CultureInfo" .

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900