Hi all
I have a folder full of CSV files (a few hundred of them each with over 100 thousand rows in) Every day we receive one of these files, and it has 45 days worth of data in it. we need to reprocess a year's worth of these files and doing them one by one would be a bloody nightmare.
I intend to output just the none overlapping section from each file and then combine it into one larger file. I'm OK with the combining, and I've done most of what looks like the difficult bit, but I'm stuck with what looks like the easiest part.
Here's my script:
#################################################
## Build up list of files and dates to combine ##
#################################################
if (test-path -path ..\filenames.csv){remove-item ..\filenames.csv}
if (test-path -path ..\filelist.csv){remove-item ..\filelist.csv}
Write-output "FileName" |out-File ..\filenames.csv
Get-ChildItem -Filter *.csv -name |sort-object name |out-File ..\filenames.csv -append
Get-ChildItem -Filter *.csv |sort-object name|
Foreach-Object{Import-Csv -path $_.Name |Where-Object {$_."Start Date" -ne''}|
Select $_.Name,@{Name="StartDateTime";Expression={$_."Start Date" + " " + $_."Start Time" -as [datetime]}} |
Measure-object StartDateTime -min -max|
Export-Csv -noTypeInformation ..\filelist.csv -append -force
}
The output:
"Count","Average","Sum","Maximum","Minimum","Property"
"115079",,,"27/08/2014 08:48:00","13/07/2014 08:56:00","StartDateTime"
"113615",,,"28/08/2014 08:51:00","14/07/2014 09:07:00","StartDateTime"
The question:
How do I also add the filename to the output? I've messed around with add-member but can't get it working at all