C: problems when parsing a file as a stream of lines

See more:

C

file

data

IO

streaming

parsing

It's pretty common that I will use fgets() in a loop to iterate through the lines of a file and process them. Normally it works well but in some cases it sucks.

Example 1: If you need to know information about the next line(s) or previous line(s) to decide how to process the current line.

Example 2: The current line indicates you have read too far and you must send it to a different context of your parser.

I know of many workarounds, but overall they tend to be untenable hacks.

I could do read the entire file in as an array of strings, and then crawl through it with relative ease.

I think this is the best solution but it comes with a major downside: memory consumption equivalent to the file size. As far as I know fopen() and fgets() are not doing that.

What are your thoughts on a best compromise? Create a file I/O interface with a small cache?

Thanks

What I have tried:

Storing two or three lines at a time and using references to process them as a set

Using an un-gets-line function to push lines back onto a stack if they have been read and need to be unread, drawing new lines off of the stack until it runs out and then fetching fresh ones.