SEC file format

Discuss the Traveller RPG and its many settings
User avatar
dorward
Mongoose
Posts: 191
Joined: Mon Aug 18, 2003 5:38 pm
Location: Tonbridge
Contact:

SEC file format

Postby dorward » Sat Jan 03, 2009 2:12 pm

Is there a definitive guide to the structure of the SEC file format?

I've found http://traveller.wikia.com/wiki/SEC_file_format, but it just gives an example without explaining what is optional, and what can change.

For example, are the column widths fixed? Or does the top section need to be parsed to identify the column widths?

It also says that "All known SEC Sector files are in the wiki, attached to each sector article under the data file link in the Infobox.", but when I go to look at them I find that they have unmentioned features.

http://traveller.wikia.com/wiki/Spinwar ... ector/data has two rows of numbers along the top. The second row seems to be used to identify where each column of data begins. The first baffles me.

Then it has a number of rows prefixed with a hash sign - presumably comments. Among these are a block of rows prefixed with a plus sign, which appear to be expansions of the political forces at work in the sector.

I could try to just make a best guess, but it would be helpful if there was a good specification for the file format somewhere. Does anyone know of one?

Thanks.
Last edited by dorward on Sun Jan 04, 2009 12:47 pm, edited 2 times in total.
Home & RPG & Geek
Beware of the Leopard
AKAramis
Greater Spotted Mongoose
Posts: 1440
Joined: Fri Aug 25, 2006 5:57 am
Location: Eagle River, Alaska, USA
Contact:

Postby AKAramis » Sat Jan 03, 2009 3:50 pm

they are position delimited files, that is, whitespace is significant. different iterations have different delimitings, but almost all have a key in the hashed comments at the top. we'll examine the sample provided on the wiki link you provided.

1-14: Name
15-18: HexNbr
20-28: UWP
31: Bases
33-47: Trade Codes & Comments
49: Zone
52-54: PBG
56-57: Allegiance
59-74: Stellar Data

This means the 1st 14 characters of a given line are the name, and padded with spaces, it takes up 14 characters EVERY TIME, within that file.

Characters 15, 16, 17 and 18 are the hex number
20,21,22,23,24,25,26, and 28 are the UWP, 27 is a dash for the TL delimiter
31 is a single character bases code.
33-47 are 3 space each trade codes.

numbers not in the listing are "visual whitespace" for human readability. the 1-14 entry should be 1-13.

So really, there isn't a definitive version, but galactic comes close.
-AKAramis
==================================================
Never catch a tiger by the tail...
... unless it is sedated or dead.
==================================================
http://aramis.hostman.us
User avatar
dorward
Mongoose
Posts: 191
Joined: Mon Aug 18, 2003 5:38 pm
Location: Tonbridge
Contact:

Postby dorward » Sat Jan 03, 2009 4:00 pm

OK, if a key is present, then it needs to be parsed. That is good to know. Thanks.

What about http://traveller.wikia.com/wiki/Spinwar ... ector/data though?
Home & RPG & Geek
Beware of the Leopard
AKAramis
Greater Spotted Mongoose
Posts: 1440
Joined: Fri Aug 25, 2006 5:57 am
Location: Eagle River, Alaska, USA
Contact:

Postby AKAramis » Sat Jan 03, 2009 7:28 pm

that 2nd line is an alternate parsing table
1 indicates 1st character sot for hex
2 for UPP
3 for bases
4 for TC's...

But tht's not traditional Sec format. it's close.
-AKAramis
==================================================
Never catch a tiger by the tail...
... unless it is sedated or dead.
==================================================
http://aramis.hostman.us
User avatar
dorward
Mongoose
Posts: 191
Joined: Mon Aug 18, 2003 5:38 pm
Location: Tonbridge
Contact:

Postby dorward » Sat Jan 03, 2009 7:29 pm

Yup, that's what I thought. It is the first line, with the 5 4 on it, that has me baffled though.
Home & RPG & Geek
Beware of the Leopard
User avatar
lastbesthope
Executive Mongoose
Posts: 19697
Joined: Sun Apr 04, 2004 4:27 pm
Location: Bristol, UK
Contact:

Re: SEC file format

Postby lastbesthope » Sat Jan 03, 2009 7:41 pm

dorward wrote:(Edit: Oh, damnation. I posted this in the General forum rather than the Traveller forum. Could a moderator move it please?)
Done

LBH
I'll live forever, heaven won't let me in and hell's afraid I'll take over!!!

Mongoose Accolades
User avatar
dorward
Mongoose
Posts: 191
Joined: Mon Aug 18, 2003 5:38 pm
Location: Tonbridge
Contact:

Re: SEC file format

Postby dorward » Sat Jan 03, 2009 7:46 pm

lastbesthope wrote:Done
Much appreciated.
Home & RPG & Geek
Beware of the Leopard
User avatar
EDG
Cosmic Mongoose
Posts: 2882
Joined: Sun Aug 12, 2007 10:28 pm
Contact:

Postby EDG » Sat Jan 03, 2009 8:00 pm

I think it's high time the sec format got updated. If only because it limits world names to 14 characters, doesn't allow for expanded UWPs, or multiple base codes (I think it's easier to just list the codes as a sequence of letters rather than force people to remember that a single letter is a combination of bases), or a lot of trade codes and comments, or really long stellar data.

I've found that something like this would work a lot better:

Code: Select all

  1-19: Name
 20-23: HexNbr
 25-33: UWP
    36: Bases
 38-52: Codes & Comments
    54: Zone
 57-59: PBG
 61-62: Allegiance
 66-100: Stellar Data
I would also like to see people adopt my bracketed stellar data as a standard - it makes things so much clearer.

Code: Select all

Stars:
* = mainworld primary
() = Close binary
Near companions are listed with no brackets.
[] = Far companion
[()] = Close binary in Far orbit
e.g.:
F8 V* [(M1 V M6)] - Mainworld orbits F8 V star, which has a close binary M1 V/M6 V system in a Far orbit.
M2 V* M6 V - Mainworld orbits M2 V star, with a Near M6 V companion.
(K4 V M8 V) - Mainworld orbits a close binary consisting of a K4 V and M8 V star. (it has to orbit both stars, because planets can't orbit the stars individually if they're Close).
M2 V* [M1 V] - Mainworld orbits an M2 V star, which has a single M1 V companion in a Far orbit.
Any post I make here may not be quoted or copied beyond the Mongoose boards without my explicit permission.
Spica forums
SFRPG Forums
AKAramis
Greater Spotted Mongoose
Posts: 1440
Joined: Fri Aug 25, 2006 5:57 am
Location: Eagle River, Alaska, USA
Contact:

Postby AKAramis » Sun Jan 04, 2009 2:13 am

EDG: many sec files had longer name fields.

The "standard" included the specification table, adjusted for the specific file.

The thing is, there really is no standard for .sec files other than
(1) "space delimited text" (positionally delimited, really).
(2) stored as pure ASCII text (7-bit or 8-bit-1st128)
(3) the nominal order is Name, Hex, UPP, Bases, trade codes, TZ, PBG, Stellar Data.
-AKAramis
==================================================
Never catch a tiger by the tail...
... unless it is sedated or dead.
==================================================
http://aramis.hostman.us
Deniable
Mongoose
Posts: 171
Joined: Thu Aug 09, 2007 4:39 am
Location: Perth, Western Australia

Postby Deniable » Sun Jan 04, 2009 12:06 pm

Most SEC files I've seen follow the same setup. The only real variation I've seen is the length of the name field and the variation is at a 'collection' rather than file level. A decent regex can handle most of them.

The best source I've found for SEC files is The Missouri Archive and specifically this directory.

As for replacing SEC, good luck. :)
tjoneslo
Stoat
Posts: 63
Joined: Mon Oct 29, 2007 11:21 pm

Re: SEC file format

Postby tjoneslo » Sun Jan 04, 2009 4:30 pm

dorward wrote:Is there a definitive guide to the structure of the SEC file format?

I've found http://traveller.wikia.com/wiki/SEC_file_format, but it just gives an example without explaining what is optional, and what can change.

For example, are the column widths fixed? Or does the top section need to be parsed to identify the column widths?

It also says that "All known SEC Sector files are in the wiki, attached to each sector article under the data file link in the Infobox.", but when I go to look at them I find that they have unmentioned features.

http://traveller.wikia.com/wiki/Spinwar ... ector/data has two rows of numbers along the top. The second row seems to be used to identify where each column of data begins. The first baffles me.

Then it has a number of rows prefixed with a hash sign - presumably comments. Among these are a block of rows prefixed with a plus sign, which appear to be expansions of the political forces at work in the sector.

I could try to just make a best guess, but it would be helpful if there was a good specification for the file format somewhere. Does anyone know of one?
The sector data files in the Traveller wiki are copies of the data files used by the NROUTE.C code, which is used to generate the Trade map summary and all of the trade maps.

The first two numbers are an x/y coordinate of the location of the sector, relative to all others. The comments at the header of the file contains the complete requirements and specifications of the nroute.c SEC file formats.
Join the Librarians, the true power in the universe: The Traveller Wiki
User avatar
EDG
Cosmic Mongoose
Posts: 2882
Joined: Sun Aug 12, 2007 10:28 pm
Contact:

Postby EDG » Mon Jan 05, 2009 3:22 am

Deniable wrote:As for replacing SEC, good luck. :)
It's not so much replacing as updating it.

After all, MGT has a Temperature parameter now (and they don't tell you how to include it in a UWP annoyingly. My preference would be a single digit column next to the PBG that goes from 1 to 5, where 1 is Frozen, 5 is Roasting, and E is extreme (for vacuum worlds that are both frozen and roasting on their night and day sides respectively). And also there's all the gov/law stuff, which may be convertible to a UWP.

I also came up with a "Supplemental" table consisting of Name, Hex location, WTN (from GT:Far Trader), "ZOT" (orbital Zone, Orbit number, Tidal lock?) and actual population (expressed as people, not a UWP).

A typical example would be this:

Code: Select all

Planet             0109    3.0   I 2 Y                   730,000
So "Planet" is located at hex 0109, has WTN of 3.0, is in the Inner Zone, Orbit #2, is Tidal-locked, and has a population of 730,000 living on it.
Any post I make here may not be quoted or copied beyond the Mongoose boards without my explicit permission.
Spica forums
SFRPG Forums
simonh
Duck-Billed Mongoose
Posts: 1527
Joined: Thu Aug 10, 2006 3:56 pm

Postby simonh » Mon Jan 05, 2009 7:11 am

A fixed field format like this is about as inflexible a file format as you can get. CSV would be more flexible, but still highly restrictive because you still wouldn't be able to support different or custom world attributes. In fact they're so inflexible I'd forget about trying to extend or revise the definition.

I'd prefer a key/value format such as classic Windows style INI files or JSON notation as most languages have decent library suport for these nowadays. A bit advantage of key/value is that it can support all sorts of optional or variant world attributes cleanly, and software that deosn't support certain attributes can just ignore them (though it should still round-trip them when writing out a modified copy of the data file).

Just blue-sky, as without an implementation this is purely hypothetical. While SEC exists and has implementations, therefore should at least be supported as an interchange format by new software, for new projects I'd deffinitely recommend moving to a more standard and flexible format.

Simon Hibbs
Deniable
Mongoose
Posts: 171
Joined: Thu Aug 09, 2007 4:39 am
Location: Perth, Western Australia

Postby Deniable » Mon Jan 05, 2009 10:50 am

You're also forgetting that SEC is human readable. It looks a lot like the lists in the printed books.

There have been multiple attempts to build/design better interchange formats but none have really succeeded. You need at least one good reference implementation and you need people to publish sectors for it. Maybe then, the format will take off.
Deniable
Mongoose
Posts: 171
Joined: Thu Aug 09, 2007 4:39 am
Location: Perth, Western Australia

Postby Deniable » Mon Jan 05, 2009 10:58 am

EDG wrote:
Deniable wrote:As for replacing SEC, good luck. :)
It's not so much replacing as updating it.
I'll still wish you luck getting it adopted. A more detailed format would be great. I'd like to see a data format with expanded system data in it, but that's not likely either.
tjoneslo
Stoat
Posts: 63
Joined: Mon Oct 29, 2007 11:21 pm

Postby tjoneslo » Mon Jan 05, 2009 3:14 pm

EDG wrote:
Deniable wrote:As for replacing SEC, good luck. :)
It's not so much replacing as updating it.
This would catch on faster if the following would happen:

1) an agreed upon new, updated format were published. Well, agreed upon may be a stretch, but at least publicly posted where anyone can find it with relative ease. If you have no objections, I'll collect the random thoughts you've posted here into a single page, and add it to the Sector File Formats listing in the Traveller Wiki

2) a collection of sector files with all the (new, updated) data. Start with the Spinward Marches. There are 35 sectors in the Imperium, 36 in Atlas of the Imperium, 92 in the wiki. So somewhere between one and 100 sectors need to be published somewhere, where fans can find and use them.

It's one thing to slightly update a well known sector (like the Spinward Marches or Solomani Rim), its another to figure out how much re-generation needs to be done in a sector like Antares or Delphi.
Join the Librarians, the true power in the universe: The Traveller Wiki
User avatar
dorward
Mongoose
Posts: 191
Joined: Mon Aug 18, 2003 5:38 pm
Location: Tonbridge
Contact:

Postby dorward » Mon Jan 05, 2009 3:41 pm

I'm trying to build a sector data to SVG graphical map tool. So I'd be interested in contributing to a new format and writing a reference implementation in Perl.
Home & RPG & Geek
Beware of the Leopard
tjoneslo
Stoat
Posts: 63
Joined: Mon Oct 29, 2007 11:21 pm

Postby tjoneslo » Mon Jan 05, 2009 7:30 pm

I'm working on a Java version of Galactic, with many new additional features. I, too would like to know of any new SEC file format, and be willing to provide input and a working program.

Currently I use XML for a data format storage. Fixed format files are nice because they are easy to read (both computers and humans) and write.
Join the Librarians, the true power in the universe: The Traveller Wiki
kristof65
Lesser Spotted Mongoose
Posts: 696
Joined: Fri Dec 12, 2008 7:04 am

Postby kristof65 » Mon Jan 05, 2009 7:32 pm

Just an idea to throw out there, given my limited knowledge of the .sec format and XML, but wouldn't XML be the "solution" to this?

As I understand, part of the XML file structure defines the data structure within in terms that any XML capable program can translate. So if someone were to convert the .sec files to XML, any one with recent versions of MS Office and Open Office (among others) could use the files, while specialty programs could be written to extract the appropriate data from any XML sector file that used the correct headings, extra data or different formatting excluded.

All that would really need to be defined by the community would be a set of "standard" tags.

Again, I may not know what I'm talking about, but it seems like that's what XML is really for.
tjoneslo
Stoat
Posts: 63
Joined: Mon Oct 29, 2007 11:21 pm

Postby tjoneslo » Mon Jan 05, 2009 7:44 pm

If you know how to read an XML Schema Document (XSD) file, I've posted a relatively complete, if old version Here as part of the online documentation for the project. There is also a CSS file to format the XML and if I spent more time with it, it could be better rendered into whatever format we wanted.
Join the Librarians, the true power in the universe: The Traveller Wiki

Who is online

Users browsing this forum: arcador, Condottiere and 28 guests