MatPlus.Net

 Website founded by
Milan Velimirović
in 2006

9:02 UTC
 
  Forum*
 
 
 
 

Username:

Password:

Remember me

 
Forgot your
password?
Click here!
SIGN IN
to create your account if you don't already have one.
CHESS
SOLVING

Tournaments
Rating lists
01-Oct-2019

B P C F





 
 
MatPlus.Net Forum Internet and Computing PBM format
 
You can only view this page!
(1) Posted by Mihail Croitor [Saturday, Apr 14, 2007 12:46]

PBM format


Hello all,
as i can see, the most of applications for organizing chess problems can open PBM(Problemiste) format. is this format open? can i find description of this format? Shure, more simple to write a letter to Matthieu Leschemelle, but, i think, answer at this question will be interested not only for me.

best wishes,
Mihail
 
(Read Only)pid=617
(2) Posted by Dmitri Turevski [Saturday, Apr 14, 2007 22:13]

Hi,

About a year ago i simply reverse engineered the pbm format when i needed to gather some stats from the problemiste collections. The file itself has a very straightforward structure, as far as i remember: first 4 bytes is the index of the last position (i.e. problem count - 1), then fixed-size blocks for each problem go (including the stipulation etc), the tail of the file consist of varying-length data (like author names, etc). Because i needed only the stipulation and the position itself, RE and parsing took like 15 minutes altogether. If you plan to create a legal utility for public domain (unlike me) i think you just *have to* contact Matthieu. If not, here are some hints that worked for me (unfortunately all my code is lost with my *samsung* hard drive and i cant share it):
- get a hex-viewer and a diff utility (i used total commander's Lister and windows alt+tab - not a best choice, definitily)
- save an empty #2, h#2, s#2, #3 to separate files
- save 2 empty #2 in one file
- save an initial array of pieces as a single file
- do it with twins, authors, awards, comments etc
- diff the files and now you know the pbm format :)
It sounds like a lot of work, but it's 15 mins really.

good luck,
Dima
 
 
(Read Only)pid=621
(3) Posted by Milan Velimirović (+) [Saturday, Apr 14, 2007 22:55]; edited by Milan Velimirović (+) [07-04-15]

I made - in last century :) - an import/export module for PBM files for my old program (which, surprisingly, I still use) MatPlus Librarian. This 16-bit Wondows program itself is now and anachronism, but at least you can get the problems converted to a plain text file format. No fairy pieces support, though...

It can be downloaded from: www.matplus.org.yu/mpl/Mplib.htm

I could supply the source code, but it is very dirty* (and I wouldn't wish even to my worst enemy to be forced to go through it). As Dmitri said it's a simple format but still took me much, much more than 15 minutes to write it... And I never completed the conversion of all possible twins notation formats - I had to ask Matthieu for his twins-coding specification which he kindly provided for me.

* Just to make it clear: a dirty source written by me, not by Matthieu (actually, I don't have anything of his code, except a small twins-related chunk)
 
   
(Read Only)pid=622
(4) Posted by Dmitri Turevski [Saturday, Apr 14, 2007 23:49]

No-no-no, 15 minutes for FEN and stipulations only, i didn't grok the twin stuff! :)

By the way, do you know that ~78% of orthodox twomovers feature the White Queen? (the only thing i remember from that test)
 
 
(Read Only)pid=623
(5) Posted by Iļja Ketris [Sunday, Apr 15, 2007 02:53]

From Matthieu:

# [offset 0] unsigned integer: #_of_problems_minus_one
# [offset 2] array of structures RECORD[#_of_problems]
# [offset 2+ (#_of_problems + 1)*(sizeof(RECORD)) strings
#
# structure RECORD:
# WORD wNumber; // first problem = 0
# BYTE bType; // 0=direct, 1=h#, 2=s#, 3=r#
# BYTE bMoves;
# DWORD dwAuthor; //*
# DWORD dwSource;
# DWORD dwDistinction;
# DWORD dwComments; // Commentaire
# DWORD dwText; // Text
# DWORD dwTwins; // Twin conditions
# DWORD dwExtraInfo; // dedicace, etc...
# BYTE bPosition[64];
# size = 96 bytes
#
# dwAuthor, dwSource etc. are offsets to respective strings
#
# Format of string:
# WORD length
# BYTE string[length]
# BYTE zero


My own findings:

Twins string is in blocks of 4 bytes, one block per twin.
The bytes go as [type][from field][to field][piece]
Not all bytes are used in all twin types, unused bytes may contain random garbage.

Apparent twin types:

[space] diagram
" remove piece
# add piece
! move piece
... there are more, but I was lazy to document the rest.

Apparent piece notation for twins:

KQRBSP kqrbsp
&@!#$% *./-,+

(I have no idea why these characters were chosen to represent letters.)

One-byte field coordinate can be converted to "c1", "e5" etc with such perl function:
function ikX2a1 ($c) { # convert one-symbol to notation ('X' => 'a1' etc)

$h = ord($c) & 0xF8;
$l = ord($c) & 0x07;
$h = 12 - ($h>>3);
$l = chr(97 + $l);
return $l . $h;
}

This was used to generate random problems from Matthieu's collection on now dormant http://a8.q.nu site.
I didn't see any bans on reverse engineering in Problemiste license (in fact, there was no explicit license to accept).
Therefore this is governed by default legislation, and most certainly, there is no such legislation in my country.

 
 
(Read Only)pid=624
(6) Posted by Vladimir Tyapkin [Sunday, Apr 15, 2007 05:00]

It is amazing to see how many people here did reverse-engineering of pbm format.

Here is some codes in addition to Ilja's post:

piece's codes(all hex numbers):
x26 - white king
x2a - black king
x22 - white queen
x2e - black queen
x21 - white rook
x2f - black rook
x24 - white knight
x2c - black knight
x23 - white bishop
x2d - black bishop
x25 - white pawn
x2b - black pawn

twin types (all hex numbers)
x20 - diagram
x21 - move a man
x22 - remove a man
x23 - add a man
x24 - remove and add
x25 - exchange

All others types seems to be not implemented.
 
   
(Read Only)pid=625
(7) Posted by Milan Velimirović (+) [Sunday, Apr 15, 2007 12:06]; edited by Milan Velimirović (+) [07-04-15]

Here is Matrhieu's twin-notation string formatting code:

#define BORD 6; //added by me

static struct {
char text[26] ;
int type ;
} DlgItem[]={
"Diagram", 0,
"%s%s->%s%s", 6,
"-%s%s", 4,
"+%s%s", 5,
"-%s%s +%s%s", 7,
"%s%s<->%s%s", 6,
"Rotation 90°", 0,
"Rotation 180°", 0,
"Rotation 270°", 0,
"Translation %s%s-%s%s", 6,
"Transl. torique %s%s-%s%s", 6,
"Miroir vertical", 0,
"Miroir horizontal", 0,
"TypePrb", 8,
"Après la clé", 0,
"Inverser couleurs", 0,
"Duplex", 0
};
char case1[3], case2[3], piece[3], Jtmp[4] ;

int GetJumeauStr (char *test)
{
int d, i, j ;
char piec2[3], piec1[3] ;

for (j=0; j<4; j++) Jtmp[j]= test[j]-32 ;
i = Jtmp[0] & 0x1F ;
d = (Jtmp[0] & 0x20) ? TRUE : FALSE ;
case1[0]='a'+Jtmp[1]%8; case1[1]='8'-(Jtmp[1]/8);
case2[0]='a'+Jtmp[2]%8; case2[1]='8'-(Jtmp[2]/8);
piece[0]=id2[Jtmp[3] ] ; piece[1]=(Jtmp[3]>BORD) ? 'N' : 'B' ; //white/black
j=piec1[0]=piec2[0]=piec1[2]=piec2[2]=0 ;
if (Jumeaux[0]==126 && i>0) { // combinated
strcpy (sTmpBfr, "puis " );
j = 5 ;
}
else if (i>0 && i<6) {
Jtmp[1]=22+((Jtmp[1]/8))*10+(Jtmp[1]%8) ;
Jtmp[2]=22+((Jtmp[2]/8))*10+(Jtmp[2]%8) ;
piec1[0]=id2[pos[Jtmp[1] ] ] ; piec1[1]=(pos[Jtmp[1] ]>BORD) ? 'N' : 'B' ;
if (piec1[0]==' ') piec1[0]=piec1[1]= '?' ;
if (i==5) {
piec2[0]=id2[pos[Jtmp[2] ] ] ; piec2[1]=(pos[Jtmp[2] ]>BORD) ? 'N' : 'B';
if (piec2[0]==' ') piec2[0]=piec2[1]= '?' ;
}
}
}

 
   
(Read Only)pid=626
(8) Posted by Uri Avner [Sunday, Apr 15, 2007 14:16]; edited by Uri Avner [07-04-15]

May I propose that for all issues concerning programming etc. a separate section should be devoted?
These matters are highly professional and make the rest of us feel a bit incompetent when we enter the Forum... (-:
I know there is a separate topic for these issues already. What I'm suggesting is a separate sector like "Magazine", "PCCC" etc.
 
   
(Read Only)pid=627
(9) Posted by Mihail Croitor [Monday, Apr 16, 2007 15:30]

Many thanks for answers!
this discussion shows me how much i am lazy :)
 
   
(Read Only)pid=629
(10) Posted by Mihail Croitor [Wednesday, Aug 29, 2007 16:29]

hi-hi-hi! :)
after working with Problemiste & MatPlus (& many other programs) i can compare all this tools. Problemiste for me is very handy utility. But it has very big minus - this program cant work with big bases (> 2000 positions). By example, Vaclav Kotesovec on his page get a link where we can download pbm file with 20 000 problems! shure, i can open this base on MatPlus (very nice program, Milan!) but search engine in MatPlus is not so good. So, a question - how i can open big bases on pbm format in Problemiste?

best wishes, Mihail
 
 
(Read Only)pid=1298

No more posts


MatPlus.Net Forum Internet and Computing PBM format