Click here to Skip to main content
15,895,667 members
Home / Discussions / C / C++ / MFC
   

C / C++ / MFC

 
AnswerRe: Efficient way to read/write file Pin
kalberts8-Jun-20 4:32
kalberts8-Jun-20 4:32 
AnswerRe: Efficient way to read/write file Pin
Joe Woodbury8-Jun-20 14:24
professionalJoe Woodbury8-Jun-20 14:24 
GeneralRe: Efficient way to read/write file Pin
kalberts8-Jun-20 20:04
kalberts8-Jun-20 20:04 
GeneralRe: Efficient way to read/write file Pin
Joe Woodbury8-Jun-20 23:25
professionalJoe Woodbury8-Jun-20 23:25 
GeneralRe: Efficient way to read/write file Pin
kalberts9-Jun-20 3:54
kalberts9-Jun-20 3:54 
GeneralRe: Efficient way to read/write file Pin
Joe Woodbury9-Jun-20 7:20
professionalJoe Woodbury9-Jun-20 7:20 
GeneralRe: Efficient way to read/write file Pin
charlieg11-Jun-20 11:53
charlieg11-Jun-20 11:53 
GeneralRe: Efficient way to read/write file Pin
kalberts11-Jun-20 19:39
kalberts11-Jun-20 19:39 
charlieg wrote:
This all boils down to requirements, and nothing has been offered by the OP except for "I need to write it fast." There is nothing to do with hardware here - it might be another requirement.
What spun off this sub-thread of the discussion was
Joe Woodbury wrote:
If the data needs to be future proofed, consider serializing using RapidJSON (which is a very fast C++ JSON library), compressing the result with LZ4 and then writing that.
"Future proof" was not stated as a requirement, but now that Joe Woodbury presented what is - in my eyes - a rather naive approach to future proofing, I chose to point out that if you want future proofing, it takes a lot more than just using a basic structure encoding that is currently fashonable.

It seems quite obvious that Joe Woodbury has never been working in the area of long time information preservation. I have a few years of experience. I know that it is not a trivial issue. When someone makes a statement that suggests "Just use JSON and LZ4, and the information is safe for the future", I think that this is so naive that it crosses the border to "fake news", and I want to correct it.

However, Joe Woodbury is not willing to accept anything that can affect the validity of his claim, calling my comments a "rant", "senseless and unproductive", that I am "mixing up" things by pointing to other important elements, that I am "just repeating the obvious".

I wrote "Be preparered for some information loss during each move". When Joe Woodbury stated "JSON is one step above key/value pairs--how would you lose information?", I went on to provide examples. Then he comes back with "You also shifted your argument; you went from you will lose information to you can lose information", and concludes "Your straw man collection is now complete!"

I don't think anything valuable will come out of a further discussion with Joe Woodbury. So I let it rest.

What regards "just document it": I have seen guides seriously suggesting a URL to visit if you have problems connecting to the Internet. I have seen document format descriptions stored electronically in the format that is described. I have seen format "documentation" that is hopelessly inadequate - having worked with the format for a long time gives the documentation some value, but often you need access to the format designer to have him explain it. I have format descriptions on 5.25" floppies. Are you able to imagine that there could be a complete breakdown of the Internet? How much of the format descriptions would then be inaccessible?

Documenting the format is a neccesary, but not sufficient provision for the data to be accessible in the future: The documentation must unconditionally be available to the person who needs to decode some data. That takes a lot more than just JSON or something similar; it takes a full storage stack all the way down to the physical medium. That could be a physical printout, on acid free paper. How many format specifications do you currently have in a printed format on acid free paper? I've got a couple printed ones, but I am not sure that the paper will survive that many years.

One of the format descriptions I have in print is for a 30+ year old format; the manufacturer went bankrupt 28 years ago and the format description was only available internally. The format was used for tens or hundreds of thousands of documents that are still residing in old archives around the contry. If someone need to access one of these documents, how would they know that they could come to me for the format descriptions? I have asked a few of my co-workers from the early 1980s if they have kept the specification document; I haven't met one that has. It could be that my copy is the last one in existence (at least in an immediately available format - the electronic version of the format was written in its own format).

So your requirement about documenting the format is satisfied. When you need to decode a document file in that format, just come to me. Problem solved. Or...?
GeneralRe: Efficient way to read/write file Pin
charlieg12-Jun-20 8:30
charlieg12-Jun-20 8:30 
QuestionDoes WaitForSingleObject timeout after 49 days if INFINITE is passed? Pin
PrafullaVedante7-Jun-20 9:02
PrafullaVedante7-Jun-20 9:02 
AnswerRe: Does WaitForSingleObject timeout after 49 days if INFINITE is passed? Pin
Greg Utas7-Jun-20 11:00
professionalGreg Utas7-Jun-20 11:00 
AnswerRe: Does WaitForSingleObject timeout after 49 days if INFINITE is passed? Pin
RedDk7-Jun-20 12:17
RedDk7-Jun-20 12:17 
AnswerRe: Does WaitForSingleObject timeout after 49 days if INFINITE is passed? Pin
CPallini7-Jun-20 20:40
mveCPallini7-Jun-20 20:40 
GeneralRe: Does WaitForSingleObject timeout after 49 days if INFINITE is passed? Pin
PrafullaVedante8-Jun-20 0:09
PrafullaVedante8-Jun-20 0:09 
GeneralRe: Does WaitForSingleObject timeout after 49 days if INFINITE is passed? Pin
CPallini8-Jun-20 1:21
mveCPallini8-Jun-20 1:21 
GeneralRe: Does WaitForSingleObject timeout after 49 days if INFINITE is passed? Pin
Joe Woodbury8-Jun-20 15:04
professionalJoe Woodbury8-Jun-20 15:04 
QuestionTranspose of a matrix Pin
Member 148492464-Jun-20 7:08
Member 148492464-Jun-20 7:08 
AnswerRe: Transpose of a matrix Pin
kalberts4-Jun-20 7:58
kalberts4-Jun-20 7:58 
AnswerRe: Transpose of a matrix Pin
CPallini4-Jun-20 20:25
mveCPallini4-Jun-20 20:25 
GeneralRe: Transpose of a matrix Pin
Member 148492464-Jun-20 23:52
Member 148492464-Jun-20 23:52 
GeneralRe: Transpose of a matrix Pin
CPallini5-Jun-20 0:04
mveCPallini5-Jun-20 0:04 
GeneralRe: Transpose of a matrix Pin
jeron15-Jun-20 4:53
jeron15-Jun-20 4:53 
QuestionHow to draw a realtime XY plot in MFC Pin
manoharbalu1-Jun-20 22:31
manoharbalu1-Jun-20 22:31 
AnswerRe: How to draw a realtime XY plot in MFC Pin
Victor Nijegorodov1-Jun-20 22:40
Victor Nijegorodov1-Jun-20 22:40 
GeneralRe: How to draw a realtime XY plot in MFC Pin
Richard MacCutchan2-Jun-20 0:15
mveRichard MacCutchan2-Jun-20 0:15 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.