If you are trying to solve a specific problem that involves CRUD (create, read, update, delete) operations on some part(s) of some files in a structured file format, chances are good that the mechanism you're going to use to solve that problem is some kind of code (programming).
If the code you're writing is in a common scripting language typically used by power users and system administrators (say, cmd.exe, Bash, and PowerShell), it's leaning to be on-topic for Super User.
If the code you're writing is in some kind of "serious programmers" context (C#, C++, Java, anything server-side, etc.), it's leaning to be on-topic for Stack Overflow.
If you're not writing code at all, but rather, using some kind of pre-built tool to manipulate the data, it's very much on-topic for Super User.
For instance: if you are using RegexBuddy to work on a regular expression, and you have a question either about RegexBuddy or about your regular expression (but not about the code that you'll write to use the regular expression), it's topical for Super User. If you want to know how to use the regular expression API in, say, Java, that's topical for Stack Overflow.
In between these extremes there are many overlaps and gray areas. In general, you will probably get better answers about the nitty-gritty of binary file formats from Stack Overflow users (to say nothing of the fact that on Stack Overflow you have more people looking at your question than on the comparatively small Super User). But, just because you get better answers there, doesn't mean it's off-topic for Super User.
In fact, it might be on-topic for Super User, but our user base might not have a great many people who are familiar enough with the subject to be able to answer efficiently without a lot of learning / googling on our part.
It's generally a bad idea to simply ask some detail about a file format without saying how you intend to apply the knowledge you learn. Any question about the internals of a file format should have a strong pragmatic/practical tilt to it, which means, tell us how you're going to be using that information. Tell us, at a high level, what exactly you are trying to do with this file. Usually you can write this information in terms of the CRUD principles, by saying "I'd like to create this or that", or "I'd like to read this or that", and then tell us where you've previously looked up the reference for XYZ file format and ran into difficulty understanding it, etc. and what code you've written so far.
But remember, it's fairly likely to get closed or migrated if your question is just a pastie of "real programmers' code" asking people to fix your problem. Conceptual questions are much more likely to be accepted here. Human-readable file formats, like XML, are also extremely on-topic for Super User, since many system administrators tend to deal with XML configuration files of various sorts, and they can be edited using a simple text editor.
To close, I should obviously mention that we are not to discuss what is considered to be topical on Stack Overflow, on Meta Super User. Stack Overflow has their topicality boundaries; Super User has our topicality boundaries; and the two are free to intersect (or not) however the respective communities please. Anything in my answer here that implies certain facts about Stack Overflow's topicality should be taken with a grain of salt, since I am not a significant participant on Stack Overflow. I can only accurately speak to the topicality of Super User. If you have a question specifically about what is topical on Stack Overflow, ask it on Meta Stack Overflow. All we can do here is affirm what Super User's topicality boundaries are.
