SEC file format

[dorward]

Is there a definitive guide to the structure of the SEC file format?

I've found http://traveller.wikia.com/wiki/SEC_file_format, but it just gives an example without explaining what is optional, and what can change.

For example, are the column widths fixed? Or does the top section need to be parsed to identify the column widths?

It also says that "All known SEC Sector files are in the wiki, attached to each sector article under the data file link in the Infobox.", but when I go to look at them I find that they have unmentioned features.

http://traveller.wikia.com/wiki/Spinward_Marches_Sector/data has two rows of numbers along the top. The second row seems to be used to identify where each column of data begins. The first baffles me.

Then it has a number of rows prefixed with a hash sign - presumably comments. Among these are a block of rows prefixed with a plus sign, which appear to be expansions of the political forces at work in the sector.

I could try to just make a best guess, but it would be helpful if there was a good specification for the file format somewhere. Does anyone know of one?

Thanks.

 

[AKAramis]

they are position delimited files, that is, whitespace is significant. different iterations have different delimitings, but almost all have a key in the hashed comments at the top. we'll examine the sample provided on the wiki link you provided.

1-14: Name
15-18: HexNbr
20-28: UWP
31: Bases
33-47: Trade Codes & Comments
49: Zone
52-54: PBG
56-57: Allegiance
59-74: Stellar Data

This means the 1st 14 characters of a given line are the name, and padded with spaces, it takes up 14 characters EVERY TIME, within that file.

Characters 15, 16, 17 and 18 are the hex number
20,21,22,23,24,25,26, and 28 are the UWP, 27 is a dash for the TL delimiter
31 is a single character bases code.
33-47 are 3 space each trade codes.

numbers not in the listing are "visual whitespace" for human readability. the 1-14 entry should be 1-13.

So really, there isn't a definitive version, but galactic comes close.

 

[dorward]

OK, if a key is present, then it needs to be parsed. That is good to know. Thanks.

What about http://traveller.wikia.com/wiki/Spinward_Marches_Sector/data though?

 

[AKAramis]

that 2nd line is an alternate parsing table
1 indicates 1st character sot for hex
2 for UPP
3 for bases
4 for TC's...

But tht's not traditional Sec format. it's close.

 

[dorward]

Yup, that's what I thought. It is the first line, with the 5 4 on it, that has me baffled though.

 

[lastbesthope]

(Edit: Oh, damnation. I posted this in the General forum rather than the Traveller forum. Could a moderator move it please?)

Done

LBH

 

[dorward]

Done

Much appreciated.

 

[EDG]

I think it's high time the sec format got updated. If only because it limits world names to 14 characters, doesn't allow for expanded UWPs, or multiple base codes (I think it's easier to just list the codes as a sequence of letters rather than force people to remember that a single letter is a combination of bases), or a lot of trade codes and comments, or really long stellar data.

I've found that something like this would work a lot better:

  1-19: Name
 20-23: HexNbr
 25-33: UWP
    36: Bases
 38-52: Codes & Comments
    54: Zone
 57-59: PBG
 61-62: Allegiance
 66-100: Stellar Data

I would also like to see people adopt my bracketed stellar data as a standard - it makes things so much clearer.

Stars:
* = mainworld primary
() = Close binary
Near companions are listed with no brackets.
[] = Far companion
[()] = Close binary in Far orbit

e.g.:
F8 V* [(M1 V M6)] - Mainworld orbits F8 V star, which has a close binary M1 V/M6 V system in a Far orbit.
M2 V* M6 V - Mainworld orbits M2 V star, with a Near M6 V companion.
(K4 V M8 V) - Mainworld orbits a close binary consisting of a K4 V and M8 V star. (it has to orbit both stars, because planets can't orbit the stars individually if they're Close).
M2 V* [M1 V] - Mainworld orbits an M2 V star, which has a single M1 V companion in a Far orbit.

 

[AKAramis]

EDG: many sec files had longer name fields.

The "standard" included the specification table, adjusted for the specific file.

The thing is, there really is no standard for .sec files other than
(1) "space delimited text" (positionally delimited, really).
(2) stored as pure ASCII text (7-bit or 8-bit-1st128)
(3) the nominal order is Name, Hex, UPP, Bases, trade codes, TZ, PBG, Stellar Data.

 

[Deniable]

Most SEC files I've seen follow the same setup. The only real variation I've seen is the length of the name field and the variation is at a 'collection' rather than file level. A decent regex can handle most of them.

The best source I've found for SEC files is The Missouri Archive and specifically this directory.

As for replacing SEC, good luck.

 

[tjoneslo]

Is there a definitive guide to the structure of the SEC file format?

I've found http://traveller.wikia.com/wiki/SEC_file_format, but it just gives an example without explaining what is optional, and what can change.

For example, are the column widths fixed? Or does the top section need to be parsed to identify the column widths?

It also says that "All known SEC Sector files are in the wiki, attached to each sector article under the data file link in the Infobox.", but when I go to look at them I find that they have unmentioned features.

http://traveller.wikia.com/wiki/Spinward_Marches_Sector/data has two rows of numbers along the top. The second row seems to be used to identify where each column of data begins. The first baffles me.

Then it has a number of rows prefixed with a hash sign - presumably comments. Among these are a block of rows prefixed with a plus sign, which appear to be expansions of the political forces at work in the sector.

I could try to just make a best guess, but it would be helpful if there was a good specification for the file format somewhere. Does anyone know of one?

The sector data files in the Traveller wiki are copies of the data files used by the NROUTE.C code, which is used to generate the Trade map summary and all of the trade maps.

The first two numbers are an x/y coordinate of the location of the sector, relative to all others. The comments at the header of the file contains the complete requirements and specifications of the nroute.c SEC file formats.

 

[EDG]

As for replacing SEC, good luck.

It's not so much replacing as updating it.

After all, MGT has a Temperature parameter now (and they don't tell you how to include it in a UWP annoyingly. My preference would be a single digit column next to the PBG that goes from 1 to 5, where 1 is Frozen, 5 is Roasting, and E is extreme (for vacuum worlds that are both frozen and roasting on their night and day sides respectively). And also there's all the gov/law stuff, which may be convertible to a UWP.

I also came up with a "Supplemental" table consisting of Name, Hex location, WTN (from GT:Far Trader), "ZOT" (orbital Zone, Orbit number, Tidal lock?) and actual population (expressed as people, not a UWP).

A typical example would be this:

Planet             0109    3.0   I 2 Y                   730,000

So "Planet" is located at hex 0109, has WTN of 3.0, is in the Inner Zone, Orbit #2, is Tidal-locked, and has a population of 730,000 living on it.

 

[simonh]

A fixed field format like this is about as inflexible a file format as you can get. CSV would be more flexible, but still highly restrictive because you still wouldn't be able to support different or custom world attributes. In fact they're so inflexible I'd forget about trying to extend or revise the definition.

I'd prefer a key/value format such as classic Windows style INI files or JSON notation as most languages have decent library suport for these nowadays. A bit advantage of key/value is that it can support all sorts of optional or variant world attributes cleanly, and software that deosn't support certain attributes can just ignore them (though it should still round-trip them when writing out a modified copy of the data file).

Just blue-sky, as without an implementation this is purely hypothetical. While SEC exists and has implementations, therefore should at least be supported as an interchange format by new software, for new projects I'd deffinitely recommend moving to a more standard and flexible format.

Simon Hibbs

 

[Deniable]

You're also forgetting that SEC is human readable. It looks a lot like the lists in the printed books.

There have been multiple attempts to build/design better interchange formats but none have really succeeded. You need at least one good reference implementation and you need people to publish sectors for it. Maybe then, the format will take off.

 

[Deniable]

As for replacing SEC, good luck.

It's not so much replacing as updating it.

I'll still wish you luck getting it adopted. A more detailed format would be great. I'd like to see a data format with expanded system data in it, but that's not likely either.

 

[tjoneslo]

As for replacing SEC, good luck.

It's not so much replacing as updating it.

This would catch on faster if the following would happen:

1) an agreed upon new, updated format were published. Well, agreed upon may be a stretch, but at least publicly posted where anyone can find it with relative ease. If you have no objections, I'll collect the random thoughts you've posted here into a single page, and add it to the Sector File Formats listing in the Traveller Wiki

2) a collection of sector files with all the (new, updated) data. Start with the Spinward Marches. There are 35 sectors in the Imperium, 36 in Atlas of the Imperium, 92 in the wiki. So somewhere between one and 100 sectors need to be published somewhere, where fans can find and use them.

It's one thing to slightly update a well known sector (like the Spinward Marches or Solomani Rim), its another to figure out how much re-generation needs to be done in a sector like Antares or Delphi.

 

[dorward]

I'm trying to build a sector data to SVG graphical map tool. So I'd be interested in contributing to a new format and writing a reference implementation in Perl.

 

[tjoneslo]

I'm working on a Java version of Galactic, with many new additional features. I, too would like to know of any new SEC file format, and be willing to provide input and a working program.

Currently I use XML for a data format storage. Fixed format files are nice because they are easy to read (both computers and humans) and write.

 

[kristof65]

Just an idea to throw out there, given my limited knowledge of the .sec format and XML, but wouldn't XML be the "solution" to this?

As I understand, part of the XML file structure defines the data structure within in terms that any XML capable program can translate. So if someone were to convert the .sec files to XML, any one with recent versions of MS Office and Open Office (among others) could use the files, while specialty programs could be written to extract the appropriate data from any XML sector file that used the correct headings, extra data or different formatting excluded.

All that would really need to be defined by the community would be a set of "standard" tags.

Again, I may not know what I'm talking about, but it seems like that's what XML is really for.

 

[tjoneslo]

If you know how to read an XML Schema Document (XSD) file, I've posted a relatively complete, if old version Here as part of the online documentation for the project. There is also a CSS file to format the XML and if I spent more time with it, it could be better rendered into whatever format we wanted.