XML

Monkey Programming Forums/User Modules/XML

Skn3(Posted 2012) [#1]
I am sure people will have some use for this. A single file xml module!

The module is pretty stable now maybe some people wouldn't mind helping to test it further?

You can find the module here:
https://github.com/skn3/xml

This written from scratch and doesn't relate to the "config" banana example that comes with monkey.

Enjoy, for free :D



slenkar(Posted 2012) [#2]
do you think its faster?

Im using the config thingy for my serialization at the moment


Skn3(Posted 2012) [#3]
Havnt compared but it uses far less string manipulation, next to none. It also lets you lookup children by matching attributes.. Which config didn't.


c.k.(Posted 2012) [#4]
I have data like this:

format_code('
<roar tick="135516477893">
<user>
<view status="ok">
<attribute type="special" ikey="id" value="12345678901234567890"/>
<attribute type="special" ikey="xp" value="0" level_start="0" next_level="20"/>
<attribute type="special" ikey="level" value="1"/>
<attribute type="special" ikey="facebook_uid" value="0"/>
<attribute type="special" ikey="name" value="cklester"/>
<attribute ikey="circuit001" label="circuit_001" value="0" type="core"/>
<attribute ikey="ct001002" label="ct_001_002" value="0" type="core"/>
<attribute ikey="ct001001" label="ct_001_001" value="1809876543" type="core"/>
<attribute ikey="premium_currency" label="Premium Currency" value="5" type="currency" min="0"/>
</view>
</user>
</roar>
')

Using your library, how would I get the "value" of the "attribute" with ikey="ct001001?"


Skn3(Posted 2012) [#5]
As long as you were sure the file would have it you could do like so:
format_code('
Local doc := ParseXML(xml_data_here)
Print doc.FindChild("user").FindChild("view").FindChild("attribute","ikey=ct001001").value
')


c.k.(Posted 2012) [#6]
NICE! I'm gonna give it a try... :-)


therevills(Posted 2012) [#7]
Cool! Nice to see some more XML parsers out there... Samah did an excellent job on the Diddy one and the speed is super fast now!

http://code.google.com/p/diddy/source/browse/trunk/src/diddy/xml.monkey


c.k.(Posted 2012) [#8]
Maybe we could get some speed tests and some collaboration amongst the XML parser providers? We all want the best.


Difference(Posted 2012) [#9]
Looking forward to trying this. The single file approach is very handy.
I'm sure I can put the special functions from my own xml module in some extra functions.

Suff like: GetOrCreateNode(nodename) and GetAttributeValue(name,defaulttothis)


c.k.(Posted 2012) [#10]
therevills, what's the diddy.xml code for the above request?


...how would I get the "value" of the "attribute" with ikey="ct001001?"




therevills(Posted 2012) [#11]
A bit more verbose:

[monkeycode] Local doc:XMLDocument = New XMLParser().ParseFile("test.xml")
Local rootElement:XMLElement = doc.Root

For Local userNode:XMLElement = Eachin rootElement.GetChildrenByName("user")
For Local viewNode:XMLElement = Eachin userNode.GetChildrenByName("view")
For Local attributeNode:XMLElement = Eachin viewNode.GetChildrenByName("attribute")
If attributeNode.GetAttribute("ikey") = "ct001001" Then
Print attributeNode.GetAttribute("value")
End If
Next
Next
Next[/monkeycode]

Of course you (we) could just a helper function do the above.


Samah(Posted 2012) [#12]
Nice! I like your idea of filtering by attributes, so I added it to the Diddy XML parser. Get the latest revision from Bitbucket.

[monkeycode]Local doc:XMLDocument = New XMLParser().ParseFile("test.xml")
Print doc.Root.GetFirstChildByName("user").GetFirstChildByName("view")
.GetFirstChildByName("attribute","ikey=ct001001").GetAttribute("value")[/monkeycode]


benmc(Posted 2012) [#13]
Is this just for reading the XML, or is it possible to update the file as well?


Samah(Posted 2012) [#14]
@benmc: Is this just for reading the XML, or is it possible to update the file as well?

Diddy's XML parser can export XML format to a string, but it's up to you to write the file (since this is not an easy thing for all targets). I haven't looked at Skn3's version to see if it can.


c.k.(Posted 2012) [#15]
Is there a way I could tell diddy.xml that I want "FindChild" (or better, "GetChild") to be a shortened alias for GetFirstChildByName()? :-D


Skn3(Posted 2012) [#16]
Hey apologies if I am stepping on any toes by the way, was not my intention. I have used the diddy XML stuff before but i just wanted something that could just be quickly imported as a flat file into simple projects.

This module has AddChild and SetAtrribute methods so you can create XML structures from scratch. Use node.Export() to retrieve an XML string which you can write to a file if you want.


Samah(Posted 2012) [#17]
c.k.: Is there a way I could tell diddy.xml that I want "FindChild" (or better, "GetChild") to be a shortened alias for GetFirstChildByName()? :-D

Nope. After doing some Objective-C coding I got into the habit of verbose method names that explain exactly what they do. If you want it to be called GetChild, feel free to fork it. Diddy is open source, after all.
https://bitbucket.org/swoolcock/diddy/fork

Skn3: Hey apologies if I am stepping on any toes by the way,

Of course not! ;)

Skn3: ...i just wanted something that could just be quickly imported as a flat file into simple projects.

The main dependencies for the Diddy parser are ArrayList and the exception handling stuff. I'll put some thought into removing dependencies but at the moment it "works". :)


Skn3(Posted 2012) [#18]
W00t, well glad to have encouraged some tweaks of code in diddy and all that!

I was going to code an xpath query method but well it wasn't really essential so settled for the simplified attribute query.

If anyone feels they want to contribute or have a tweak then feel free to post it or do via git and I can update the module accordingly.


AdamRedwoods(Posted 2012) [#19]
SKN3.XML failed the test.
Diddy.XML passed ok, but PLEASE return an empty Child rather than a null on nodes not found. (hard crash).

format_codebox('
#TEXT_FILES ="*.dae"

#USE_DIDDY = 0

Import mojo
#If USE_DIDDY = 1
Import diddy
#Else
Import skn3.xml
#Endif



Function Main()

New MyGame()

End


Class MyGame Extends App


Method OnCreate()

SetUpdateRate 30
Local object_id$ = ""
Local geom_id$ = ""

#If USE_DIDDY = 1
Local doc:XMLDocument

Try
doc = New XMLParser().ParseFile("duck.dae")
Catch ex:Throwable
Print "** file not found"
Print DiddyException(ex).Message()
End

If Not doc
Print "** file not found"
Else
Print "..parsed"
'Print doc.Root.GetFirstChildByName("library_visual_scenes").GetFirstChildByName("visual_scene").GetFirstChildByName("node").GetFirstChildByName("instance_geometry").GetAttribute("url")
object_id = doc.Root.GetFirstChildByName("library_visual_scenes").GetFirstChildByName("visual_scene").GetFirstChildByName("node").GetFirstChildByName("instance_geometry").GetAttribute("url")

Print ".."
geom_id = doc.Root.GetFirstChildByName("library_geometries").GetFirstChildByName("geometry").GetAttribute("id")

Endif

#Else
Local err:XMLError = New XMLError
Local doc:XMLDoc = ParseXML(LoadString("duck.dae"), err)

If Not doc
Print "** file not found"
If err And err.error Then Print err.message
Else
Print "..parsed"
object_id= doc.FindChild("COLLADA").FindChild("library_visual_scenes").FindChild("visual_scene").FindChild("node").FindChild("instance_geometry").GetAttribute("url")
Print ".."
geom_id = doc.FindChild("COLLADA").FindChild("library_geometries").FindChild("geometry").GetAttribute("id")


Endif
#endif

If object_id Then Print object_id
If geom_id Then Print geom_id

End

Method OnRender()

End

Method OnUpdate()

End

End
')


duck.dae
https://collada.org/owl/browse.php?sess=0&parent=126&expand=1&order=name&curview=0


Difference(Posted 2012) [#20]
"PLEASE return an empty Child rather than a null on nodes not found. (hard crash)."


But if I save the XML after that call, is there then a new child in the doc?
I would not want a reference to a child node that was not in the XML tree.
Also, I would not like a parser to add nodes, unless I tell it to.
To me, returning null, if a node does not exist, seems like correct behavior.


AdamRedwoods(Posted 2012) [#21]
Good points-- I was thinking that the parser could return a null node or error node, which has no value, no children, no parent, etc. Just handled differently so it would not hard crash on an element not found.


Skn3(Posted 2012) [#22]
Maybe I could have a read only flag for nodes and then keep a readonly global null node (unattached) per doc and return that? Hows that sound?


Skn3(Posted 2012) [#23]
I have updated the module on the repo. Added lots of little bits:

' - changed Find___ functions to Get___
' - added GetDescendants() for getting all descendants of node
' - add path lookup ability see GetChildAtPath() and GetChildrenAtPath()
' - added default null return nodes so function chaining wont crash app
' - made it so node can be readonly, used for the default null node
' - added GetParent() for safe traversal of teh node strucutre
' - added special @value into query string to look for a nodes value


I also updated the example file and it shows some more use cases.

format_codebox('Local error:= New XMLError
Local doc:= ParseXML(LoadString("test2.xml"), error)
If doc = Null and error.error
'error
Print error.ToString()
Else
'success
'get all books
Print "[get title of all books]"
Local nodes:= doc.GetDescendants("title")
For Local node:= EachIn nodes
Print node.value
Next
Print ""

'get all fantasy books
Print "[get all fantasy books]"
nodes = doc.GetDescendants("genre", "@value=Fantasy")
For Local node:= EachIn nodes
Print node.GetParent().GetChild("title").value
Next
Print ""

'get null book
Print "[get null attribute from null node]"
Print doc.GetChild("this_doesnt_exist").GetAttribute("some_value", "default_value")
Print ""

'print node paths for all books by Corets, Eva
Print "[get paths to all books by Corets, Eva]"
nodes = doc.GetDescendants("author", "@value=Corets, Eva")
For Local node:= EachIn nodes
Print node.GetParent().GetChild("title").value + " at '" + node.GetParent().path + "'"
Next
Print ""

'get node at path
Print "[get description of first book at path book/description]"
Print doc.GetChildAtPath("book/description").value
Print ""

'get node at path with attributes
Print "[get title of first book at path book/genre where genre value is Fantasy]"
Print doc.GetChildAtPath("book/genre", "@value=Fantasy").GetParent().GetChild("title").value
Print ""
EndIf')


Skn3(Posted 2012) [#24]
AdamRedwoods, Re failed test:
I made it so the module treats doc also as the root node so you dont need to do doc.FindChild("COLLADA") as doc will infact be the COLLADA node.


Skn3(Posted 2012) [#25]
Ok updated again, probably last one now apart from bug fixes.

I switched the parsing of opening tags from raw.Find("<?xml",offset) to HasStringAtOffset("<?xml",raw,offset) and the parsing is speedy fast now.

I tested a 100k file purely for parsing and here are the results:
diddy - 33ms
xml - 36ms
config - 100ms

So diddy is the fastest but this xml module is doing a bit more under the hood so that is to be expected.


Samah(Posted 2012) [#26]
@AdamRedwoods: Good points-- I was thinking that the parser could return a null node or error node, which has no value, no children, no parent, etc. Just handled differently so it would not hard crash on an element not found.

I might do something like this. To be honest I'd rather it throw an exception to be caught. Null return seems to be a better option.

Edit:
@Skn3: ...but this xml module is doing a bit more under the hood so that is to be expected.

What else is it doing?


Skn3(Posted 2012) [#27]
Just updated with a small tweak. instead of classing a node as 'readonly' when its null.. i switched it over to 'valid' as this makes more sense in practice.

So for example to test a node exists you would do:
format_code('If doc.GetChild("node").valid = False Print "node doesn't exist!"')

What else is it doing?

Not much more but little things like its tracking line/column numbers, it builds lists of nodes at certain paths as it parses and from what I can see has slightly more error checking/reporting of the xml.

Only very minor differences hence the slight increase of ms.


Samah(Posted 2012) [#28]
@Skn3: ...tracking line/column numbers...

I actually have a local version of the Diddy parser that does just that, for things like "missing bracket at line 3, column 7". I wrote it ages ago but I never committed it because it's not quite complete and I had more interesting things to do (like the storyboard module).


Skn3(Posted 2012) [#29]
You should upload it, it helps avoid some of those long bug hunts further down the line.

Just a small update on my module. I have added GetNextSibling() and GetPreviousSibling methods so a node can search for next/prev siblings that match tagname or attributes.

Example usage:
format_codebox(''load xml
Local error:XMLError
Local xml:= ParseXML(path, error)
If xml = Null Return Null

'parse xml
Local filter:String = "genre=action"
Local item:XMLNode = xml.GetChild("item",filter)
While item
item = item.GetNextSibling("item",filter)
Wend')


c.k.(Posted 2012) [#30]
format_code('
<roar tick="0">
<leaderboards>
<list status="ok">
<board board_id="3012" ikey="circuit001" resource_id="3012" label=""/>
<board board_id="3676" ikey="ct001002" resource_id="3676" label=""/>
<board board_id="3888" ikey="ct001001" resource_id="3888" label=""/>
</list>
</leaderboards>
</roar>
')

I need a list of ikeys and resource_ids from the above XML. How do I extract it, and what's the best way to store it? String[]? ArrayList<Leaderboard>?


Skn3(Posted 2012) [#31]
You could do:
format_code('Local xml := ParseXML(xml_data)
Local board := xml.GetChildAtPath("leaderboards/list/board")
While board.valid
'Get data here
Print board.GetAttribute("ikey")
Print board.GetAttribute("resource_id")

'Next board
board = board.GetNextSibling("board")
Wend')

If you do it this way it won't create any garbage objects for iterating and you won't have to create a temporary list object (more garbage) which you would have if you called GetChildren("board").

I'd probably store each record as a custom class stored in an array as I imagine new items won't be added?


c.k.(Posted 2012) [#32]
Nice! Thank you.

Yes, I think I'm going with the custom class stored in an array.


DGuy(Posted 2012) [#33]
Bug:
If element name has upper-case characters ParseXML fails with error: "mismatched end tag".

Cause:
As nodes are created from element names, the names are converted to lower-case, but when checking a newly parsed closing-tag-name against its' parents' name, the closing-tag-name is not first converted to lower case, leading to the error.

Fix:
Call 'ToLower()' when retrieving closing-tag-name (~line 1635):
format_code('
'this is a closing tag
tagName = attributeBuffer.value.ToLower() ' <-- FIX: Added 'ToLower()'

'check for mismatch
If parent = Null or tagName <> parent.name
'error
If error error.Set("mismatched end tag", rawLine, rawColumn, rawIndex)
Return Null
EndIf
')


Skn3(Posted 2012) [#34]
Thanks, will fix this when I get back home after crimbo holidays :)


Skn3(Posted 2013) [#35]
This fix has now finally been added! thanks David.


Skn3(Posted 2013) [#36]
Another little update today:

' - added GetChild() (with no name or attributes) this will allow to get first child of a node

' - added SetAttribute() and GetAttribute() overloads for bool,int,float, string and no value so don't have to do value conversion in user code!


Rushino(Posted 2013) [#37]
Exactly what i needed. Will have a look!


Midimaster(Posted 2013) [#38]
I tried to parse a (official) music notation xml file with your XML module, but I already get massiv erros:


format_code('took 10
XMLError: illegal character [line:2 column:2 offset:42]')


format_code('<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE score-partwise PUBLIC "-//Recordare//DTD MusicXML 1.1 Partwise//EN" "http://www.musicxml.org/dtds/partwise.dtd">
<score-partwise>
<part-list>
<score-part id="P1">
<part-name>part-1</part-name>
</score-part>
</part-list>
</score-partwise>
')

...and again in line 3 with the "-" sign! What can I do?


Skn3(Posted 2013) [#39]
Hey there midimaster,

I updated the module to support '-' in tags and attribute ids. Let me know how you get on please :)

[edit]
just realised that it was still pointing to github repo, please note I have moved to bitbucket.


Skn3(Posted 2013) [#40]
Added new method to node:

node.CountChildren()
node.CountChildren("tag_name")
node.CountChildren("tag_name","attributes=1")

This will count children of the node which match the given params.


Chroma(Posted 2013) [#41]
Skn3, where's the latest XML? I tried the link from your site and it doesn't work. And I just did a very simple .xml file as below.

format_code('
<book>
<title>Haberdash</title>
</book>
')

Then I did this:
format_code('
Local error:= New XMLError
Local ms:Int = Millisecs()
Local doc:= ParseXML(LoadString("test.xml"), error)
Print "took " + (Millisecs() -ms)
If doc = Null And error.error
'error
Print error.ToString()
Else
Print "Get title of the book"
Print doc.GetChildAtPath("book/title").value
Endif
')

Doesn't work for me. And also says that Error is not found perhaps you meant "error". The work Error is automatically changing to uppercase now so apparently it's a keyword.


Chroma(Posted 2013) [#42]
Yeah this is definitely broken... :(


therevills(Posted 2013) [#43]
Yeah Error is a Monkey keyword, try changing your variable to something else (although I think its a Monkey bug and that you should be able to use "error").


Skn3(Posted 2013) [#44]
Hmm how about in my signature? I will check what's happened when I get back to a pc.


Chroma(Posted 2013) [#45]
Yep that's the link where I got it from. I did change error to err and it still wouldn't work btw.


therevills(Posted 2013) [#46]
Works here Chroma... maybe tell us what error you are getting...

test/test.build/test.xml
format_code('<book>
<title>Haberdash</title>
</book>')
test/test.monkey
format_code('Import mojo
Import skn3.xml

Function Main:Int()
Local error:= New XMLError
Local ms:Int = Millisecs()
Local doc:= ParseXML(LoadString("test.xml"), error)
Print "Time Taken " + (Millisecs() - ms)
If doc = Null And error.error
Print error.ToString()
Else
Local nodes:= doc.GetDescendants("title")
For Local node:= Eachin nodes
Print node.value
Next
Print "Get title of the first book"
Print doc.GetChildAtPath("title").value
Endif

Return 1
End')
Console output:
format_code('TRANS monkey compiler V1.46
Parsing...
Semanting...
Translating...
Building...
Done.')
HTML5 Output:
format_code('Time Taken 22
Haberdash
Get title of the first book
Haberdash')


Chroma(Posted 2013) [#47]
Skn3, I have data between two tags ie <body> There is an indent here.</body>. But the 5 space indent is disappearing when I load it. I set the whitespace Constant to 0 and it's still stripping it away. Any ideas?


Skn3(Posted 2013) [#48]
Chroma,

Before I start on this.. could you explain if and how you are getting any errors still, and how I can reproduce it? Once I get that reply, I'll sit down and fix the bug + fix the white space trim bug.

Cheers.


Chroma(Posted 2013) [#49]
Come to find out it was how a string loaded with LoadString works. Now AFAIK, everything in the XML module works fine.


Skn3(Posted 2013) [#50]
W00t less work for me :))


Rushino(Posted 2013) [#51]
So basically this module can help export xml too?


Skn3(Posted 2013) [#52]
It certainly can, just use the AddChild() and SetAttribute() methods.

To get the xml data as a string you would use the Export() method.


Rushino(Posted 2013) [#53]
Nice then. I will consider using this module to handle my map format and then to make my generic world builder mixed with my Challenger GUI implementation.

Thanks for making this!


Difference(Posted 2013) [#54]
Why is this failing with a "Null object access" ?

I'm expecting it to return nullnode with .valid = False

format_code('
Local doc:= New XMLDoc("test")
Local nod:=doc.GetChild("nonodesyet")
')


Difference(Posted 2013) [#55]
Inserting : doc = Self in XMLDoc New() fixes it, but I no idea if I'm breaking anything .... ?
format_code('
Class XMLDoc Extends XMLNode
....

Method New(name:String, version:String = "", encoding:String = "")
.....
doc = Self
')


Skn3(Posted 2013) [#56]
Ah good catch, thanks. Repo has been updated.

The doc was not storing a pointer to itself in the base class XmlNode.

[edit] Seems I should have waited for the fix to be done for me hah :D ... cheers


Difference(Posted 2013) [#57]
Thanks. :-)

This is actually why I'm swithing from my own XML module to yours.
I'm thinking more people will give a common module a good thrashing, thusly finding more glitches/bugs.


Rushino(Posted 2013) [#58]
I like standalone modules like this one


Rushino(Posted 2013) [#59]
Sk3n: Would have been nice to have a simple little example on how to use AddChild, etc. to create an XML because so far i get problems.

It crash here.. child.pathList = doc.paths.Get(child.path)

Memory Exception.. probably due to pathlist to be null or child path

EDIT: Nevermind got it to work. But i think AddChild second parameter (attributes) doesn't work well. When i use the query such test=aaa&test2=bbb it crash.


Rushino(Posted 2013) [#60]
Sk3n: Would have been nice to have a simple little example on how to use AddChild, etc. to create an XML because so far i get problems.

It crash here.. child.pathList = doc.paths.Get(child.path)

Memory Exception.. probably due to pathlist to be null or child path

EDIT: Nevermind got it to work. But i think AddChild second parameter (attributes) doesn't work well. When i use the query such test=aaa&test2=bbb it crash.


Rushino(Posted 2013) [#61]
Ops.. sorry for double post.. seem like a forum bug


Skn3(Posted 2013) [#62]
Hey if you post an example of how to reproduce it I can test and fix it next time at a pc.


Rushino(Posted 2013) [#63]
Or maybe its me that doesn't know how it work.. What is the purpose of attributes as a string in AddChild second parameter ?


Skn3(Posted 2013) [#64]
Hey, it should let you set attributes with a query string. E.g. .AddChild("setting","id=setting1&value=123")

It's basically a shortcut so you don't have to call lots of set attribute calls upon node creation.

Just had a look at source on bit bucket (still not at pc) and can't see any bugs although there could well be something.

If you post a way in which it crashes I can reproduce and fix.


Rushino(Posted 2013) [#65]
As i though, well i just used it the way you did and it crashed i didn't do anything fancy.. strange.


Skn3(Posted 2013) [#66]
Update!

Fixed your bug Rushino thanks. I was referencing the array of query items directly when I should have been reference something else.

Also added two new export flags:
XML_STRIP_NEWLINE export flag which will add line feeds to exported data.

XML_STRIP_CLOSING_TAGS export flag so that xml nodes with no children would get exported as <tag /> instead of <tag></tag>


And finally I added a second example that demonstrates how to create an XML doc.


Rushino(Posted 2013) [#67]
Thanks to you :)


AdamRedwoods(Posted 2013) [#68]
fyi - i've been using this for collada files and it's held up nicely, but when i start hitting 3MB-16MB massive collada files, i hit the memory violation wall. mostly parsing vertex data in XMLStringData in the Add method
format_code('If count = data.Length data = data.Resize(data.Length + chunk)')
this line crashes out.

If i change these lines in ParseXML to increase the chunks, it works better.
format_code('
Local whitespaceBuffer:= New XMLStringBuffer(1024)
Local attributeBuffer:= New XMLStringBuffer(1024)
')

and i'm able to load this file in minib3d:
http://sketchup.google.com/3dwarehouse/details?mid=cf5e13277f3612f739e220ac20393eb5&prevstart=0

...and even this model at 16MB!
http://sketchup.google.com/3dwarehouse/details?mid=eb36760849ae2e371b3ba5fca0675bd0&prevstart=0

pretty cool.


Skn3(Posted 2013) [#69]
w00t I'm glad it is performing well with large files!

I have added your tweak to the repo, thanks :D


Skn3(Posted 2013) [#70]
Updated repo, anyone following this please update your link.


Skn3(Posted 2013) [#71]
Just an update for tidyness.

Please could anyone using the lib change their import location to "import xml" and move it into the modules root.

So it will now become "Import xml"


devolonter(Posted 2013) [#72]
[edit]Oops! Wrong thread. Sorry![/edit]


bram(Posted 2013) [#73]
Really neat module, thanks for sharing!


Why0Why(Posted 2013) [#74]
I think it was a combination of problems. I guess the updated XML and the XML has to be in the data folder. Got it going.


SHiLLSiT(Posted 2013) [#75]
I'm having issues with this module on iOS, but it works perfectly in Flash

When I go to test my iOS version, it eventually crashes (if not right away) and points to a line with m_firstChild with the error "EXC_BAD_ACCESS" (It's inside the GetChild function).

From my understanding, this error is thrown when an object that has already been released was trying to be accessed. I'm not very familiar with XCode and ObjectiveC, so I'm having a really hard time tracking this down. Any help would be greatly appreciated.


Skn3(Posted 2013) [#76]
Do you have some test code?


SHiLLSiT(Posted 2013) [#77]
Here's the class that handles loading XML levels with the XML module:
format_code('
Strict
Import monkeypunk.world
Import monkeypunk.xml.xml
Import monkeypunk.mp

Import src.props.rock
Import src.props.coin
Import src.props.ringbasic
Import src.props.ringmove
Import src.props.balloon
Import src.props.balloonmine

Class ChunkManager

Private
Global _chunkData:XMLDoc
Global _chunks:StringMap<Stack<XMLNode>>
Global _lastChunk:Int = 0
Global _chunksSinceLastCoin:Int = 0

Public
Global lastChunkName:String = ""

Function Init:Void (xmlPath:String)
_chunkData = ParseXML(LoadString(xmlPath))
_chunks = New StringMap<Stack<XMLNode>>

For Local chunk:XMLNode = Eachin _chunkData.children
' push chunk on stack
If (chunk.GetAttribute("isCoinChunk") = "True")
' if stack does not already exist, create one
If (Not _chunks.Contains(chunk.GetAttribute("difficulty") + "_coin"))
_chunks.Set(chunk.GetAttribute("difficulty") + "_coin", New Stack<XMLNode>())
Endif

' if has coins, put in coin stack
_chunks.Get(chunk.GetAttribute("difficulty") + "_coin").Push(chunk)
Else
' if stack does not already exist, create one
If (Not _chunks.Contains(chunk.GetAttribute("difficulty")))
_chunks.Set(chunk.GetAttribute("difficulty"), New Stack<XMLNode>())
Endif

' otherwise put in normal stack
_chunks.Get(chunk.GetAttribute("difficulty")).Push(chunk)
Endif
End For
End

Function LoadChunk:Void (difficulty:String)
Local index:Int
Local chunk:XMLNode
If (_chunksSinceLastCoin > GC.CHUNK_COIN_INTERVAL And MP.RandomFloat() <= GC.CHUNK_COIN_CHANCE)
' pick a random coin chunk
index = MP.RandomInt(_chunks.Get(difficulty + "_coin").Length())
_chunksSinceLastCoin = 1

' get chunk
chunk = _chunks.Get(difficulty + "_coin").Get(index)
Else
' pick a random chunk
index = MP.RandomInt(_chunks.Get(difficulty).Length())
_chunksSinceLastCoin += 1

' if chose last chunk, choose again
While (index = _lastChunk)
index = MP.RandomInt(_chunks.Get(difficulty).Length())
Wend
_lastChunk = index

' get chunk
chunk = _chunks.Get(difficulty).Get(index)
Endif

lastChunkName = chunk.GetAttribute("file")

' load props
Local world:World = MP.GetWorld()
Local props:List<XMLNode> = chunk.GetChild("props").children
Local x:Int, y:Int
For Local prop:XMLNode = Eachin props
x = Int(prop.GetAttribute("x")) + MP.width
y = Int(prop.GetAttribute("y"))

' NOTE: XML plugin lowercases all node names
Select (prop.name)
Case "ringbasic"
world.Add(RingBasic.Create().Init(x, y))
Case "ringmove"
Local nodes:Stack<Point> = New Stack<Point>()
nodes.Push(New Point(x - MP.width, y))
For Local n:XMLNode = Eachin prop.children
nodes.Push(New Point(Int(n.GetAttribute("x")), Int(n.GetAttribute("y"))))
End For

world.Add(RingMove.Create().Init(x, y, Int(prop.GetAttribute("speed")), nodes))
Case "coin"
world.Add(Coin.Create().Init(x, y))
Case "balloon"
world.Add(Balloon.Create().Init(x, y))
Case "balloonmine"
world.Add(BalloonMine.Create().Init(x, y))
Case "rock_a"
world.Add(Rock.Create().Init(x, y, prop.name))
Case "rock_b"
world.Add(Rock.Create().Init(x, y, prop.name))
Case "rock_c"
world.Add(Rock.Create().Init(x, y, prop.name))
Case "rock_d"
world.Add(Rock.Create().Init(x, y, prop.name))
End Select

End For
End

End Class
')

Our XML file is quite large, so here's a small snippet of it:
format_code('
<chunkDB>
<chunk file="easyChunk01.oel" width="2048" height="1536" difficulty="EASY" isCoinChunk="False">
<props>
<ringBasic id="4" x="912" y="736" />
<rock_a id="0" x="112" y="976" />
<rock_c id="1" x="1264" y="832" />
</props>
<background>
<background id="0" x="0" y="0" />
</background>
</chunk>
<chunk file="easyChunk02.oel" width="2048" height="1536" difficulty="EASY" isCoinChunk="False">
<props>
<ringBasic id="2" x="944" y="224" />
<rock_b id="0" x="784" y="976" />
</props>
<background>
<background id="0" x="0" y="0" />
</background>
</chunk>
<chunk file="easyChunk03.oel" width="2048" height="1536" difficulty="EASY" isCoinChunk="True">
<props>
<balloon id="7" x="528" y="384" />
<balloon id="31" x="1264" y="384" />
<coin id="0" x="944" y="1008" />
<coin id="1" x="1024" y="1008" />
<coin id="2" x="1104" y="992" />
<coin id="3" x="864" y="992" />
<coin id="4" x="784" y="976" />
<coin id="5" x="1184" y="976" />
<coin id="6" x="1264" y="960" />
<coin id="8" x="704" y="960" />
<coin id="9" x="1344" y="928" />
<coin id="10" x="624" y="928" />
<coin id="21" x="944" y="928" />
<coin id="22" x="1024" y="928" />
<coin id="23" x="1104" y="912" />
<coin id="24" x="864" y="912" />
<coin id="25" x="784" y="896" />
<coin id="26" x="1184" y="896" />
<coin id="27" x="1264" y="880" />
<coin id="28" x="704" y="880" />
<coin id="29" x="1344" y="848" />
<coin id="30" x="624" y="848" />
</props>
<background>
<background id="0" x="0" y="0" />
</background>
</chunk>
...
</chunkDB>
')


Skn3(Posted 2013) [#78]
Hmm well I had a look over the code and over teh XML module and at first glance I can't see anything that might cause it.

I do notice you are holding onto the XMLNode in your init function. Just as an experiment, what if you create a duplicate of the XMLNode retrieved from the document and store that instead? I can't really see why this would be an issue though as in the code you are not freeing the XML document.

Do you have an example code that I can run and test it on my machine?

Also do you have the full details of where it is crashing? What monkey line?

Try enabling this in xcode:
http://stackoverflow.com/questions/5386160/how-to-enable-nszombie-in-xcode
You may have to hunt around for the setting a bit depending on your xcode version. It will give you a bit more info on the exc_bad.


Skn3(Posted 2013) [#79]
Oh also noticed something in your code.

You are accessing the xml nodes without checking they actually exists. There is a mechnaism in place to prevent this being an issue, but as a rule you should either change your loop mechanism to use the built in previous/next pointers or make sure you always check If myNode.valid before doing anything to myNode.

This is the alternative looping method:
format_code('prop = chunk.GetChild("prop")
While prop.valid
'next
prop = prop.GetNextSibling("prop")
Wend')


SHiLLSiT(Posted 2013) [#80]
Hm, my problem is that I can't get the nodes under "props" with the get child method which is why I was looping through the children. So your alternative looping method actually breaks the game, since it isn't loading the individual props.

I'm currently not on the Mac, but here's the section of your module where it breaks:
format_code('
Method GetChild:XMLNode()
' --- gets the first child ---
'skip
If firstChild = Null Return doc.nullNode // breaks here

'return
Return firstChild
End
')


Skn3(Posted 2013) [#81]
Well I think perhaps there is an issue somewhere else as nullnode will always exist. Are you sure your not freeing the doc somewhere else?


SHiLLSiT(Posted 2013) [#82]
Not to my knowledge...this is the only class that uses your XML module.


Skn3(Posted 2013) [#83]
I can't really do much as I am away from pc for the next week. One suggestion, try disabling the garbage collection in monkey? What happens then?


SHiLLSiT(Posted 2013) [#84]
How would I go about turning off the garbage collector?

EDIT:
Here are some screenshots of the errors...however it seems to be breaking on a different line today.

Screenshot 1

Screenshot 2


SHiLLSiT(Posted 2013) [#85]
So after thinking about the issue and doing some tests, I've been able to make some progress.

I was able to figure out how to get the loop you suggested with the following code:

format_code('
Local prop:XMLNode = _chunk.GetChild("props").GetChild()
While (prop.valid)
// ... logic here ...
prop = prop.GetNextSibling()
Wend
')

I'm still getting the same error and the line it breaks on still isn't consistent - sometimes it will break right away while other times it takes a few moments.

The common thing I've noticed between the lines it breaks on is that its trying to access a member of the current "chunk" XML node. As I mentioned earlier, from what I've read the error (which is always "EXC_BAD_ACCESS") is caused when object is null when its trying to be accessed.

This leads me to believe it might indeed be Monkey's GC clearing the node reference. So my question is now, how do I debug this? Is there a way to disable the garbage collector?

EDIT: okay this might actually be an issue in the version of monkey that I am using (v70g) as I downloaded the newest version (v72b) and the error seems to be gone. However now I'm getting the error "signal SIGABRT" at:

format_code('
int main( int argc,char *argv[]){
NSAutoreleasePool *pool=[NSAutoreleasePool alloc] init];
UIApplicationMain(argc,argv,nil,nil); // break here
[pool release];
return 0;
')

When I disable the chunk loading (which uses the XML module) the game runs error free.


SHiLLSiT(Posted 2013) [#86]
Problem solved!

Turns out the random number generator I wrote was occasionally returning negative values, which were being used to get an index in the stack, hence the null objects.

My apologies for blaming the module.


Skn3(Posted 2013) [#87]
Wewt! Well that's good you fixed it. Always good to iron out any bugs if they are there in the XML module.


Raul(Posted 2013) [#88]
Hey there,

I have an issue here. It works perfectly on Android, but on Glfw and iOS I receive:

XMLError: unexpected end of xml [line:1 column:1 offset:0]

and this is the XML file:
format_code('
<thelist>
<Session>
<Title>Antrenament 1</Title>
<Picture>0</Picture>
<Description>Acesta este cel mai utli antrenament ever.</Description>
<Duration>70</Duration>
<NumberOfPictures>67</NumberOfPictures>
</Session>
<Session>
<Title>Antrenament 2</Title>
<Picture>1</Picture>
<Description>Acesta este un antrenament care trebuie cumparat.</Description>
<Duration>90</Duration>
<NumberOfPictures>67</NumberOfPictures>
</Session>
</thelist>
')


Skn3(Posted 2013) [#89]
Hey Raul,

Just looked at this. Check the repo for example4. This works perfectly for me on glfw?

I updated the module with a very basic error check if the xml data is zero length.

Are you sure you something else is not going on in your code such as a typo in filename or empty file? Maybe the file is not copying across? Do you have weird characters in your filename?

Can you verify that the data you are sending to ParseXML is there? Try printing it out before ParseXML is called?


Raul(Posted 2013) [#90]
hello Skn3,

it is working fine on both OS. retested with the previous version of XML module.

thanks anyway for diving into this.


Skn3(Posted 2013) [#91]
Hurrah no problem :)


Skn3(Posted 2013) [#92]
Updated repo:
'version 18
' - added GetAttributeOrAttribute() to XMLNode. This allows you to get attribute by id. If that attribute doesnt exist it looks for second id. If neither exist, default value is returned.
' - fixed null node returns in Get Previous/Next sibling methods


This allows you to do something like

format_code('Local padding:String = node.GetAttributeOrAttribute("padding_left","padding","<null>")')
In this line it would first try to get the attribute "padding_left". If that was not defined in your xml it would then try to get "padding". Finally if neither are defined, a default value of "<null>" is returned.


computercoder(Posted 2013) [#93]
SKN3:

I created an overload called "GetChildren:List<XMLNode>()" that may be useful to others for grabbing ALL children of a node.

In the "XMLNode" class:
format_code('
Method GetChildren:List<XMLNode>()
' --- get all children ---
Local result:= New List<XMLNode>

'skip
If firstChild = Null Return result

'scan children
If firstChild <> Null
Local child:= firstChild
While child
'Add child
result.AddLast(child)

'next child
child = child.nextSibling
Wend
Endif

'return the result
Return result
End
')

I needed this so I could iterate all of the children for a generic way to read what is in a given node. Someone else may find it useful as well :)


computercoder(Posted 2013) [#94]
SKN3:

I was working with the XML writer portion of this and found an issue with the internal Export method of XMLNode.

If I have the XML_STRIP_CLOSING_TAGS option on AND there are no children nodes AND I have a value for the node, the current code fails to write the value to the buffer.

Use this example to test against the current code:
format_code('
Import mojo
Import xml

'--- test program---
Class TestApp Extends App
Method OnCreate:Int()
Local error:= New XMLError

'create a doc
Local doc:= New XMLDoc("Test","1.0","UTF-8")

'create a group
Local group:= doc.AddChild("MyChild")
group.value = "I have a value!"

'print then resulting XML
Print doc.Export(XML_STRIP_CLOSING_TAGS)

'quit the app
Error ""
Return True
End
End

Function Main:Int()
New TestApp
Return True
End
')
It would look like this:
format_code('
<?xml version="1.0" encoding="UTF-8"?>
<test>
<mychild />
</test>
')
Notice its missing the value "I have a value!" in the "data" tag. It truncates the value AND shortens the tag!

I corrected this issue with the following code:
format_codebox('

'internal
Private
Method Export:Void(options:Int, buffer:XMLStringBuffer, depth:Int)
' --- convert the node to a string ---
'make sure there is a buffer to work with
If buffer = Null buffer = New XMLStringBuffer(1024)

Local index:Int

'add opening tag
'ident
If options & XML_STRIP_WHITESPACE = False
For index = 0 Until depth
buffer.Add(9)
Next
EndIf

buffer.Add(60)
buffer.Add(name)

'add attributes
For Local id:= EachIn attributes.Keys()
buffer.Add(32)
buffer.Add(id)
buffer.Add(61)
buffer.Add(34)
buffer.Add(attributes.Get(id).value)
buffer.Add(34)
Next

'check for short tag
If children.IsEmpty() And options & XML_STRIP_CLOSING_TAGS And Not value.Length
'no children so short tag
'finish opening tag
buffer.Add(32)
buffer.Add(47)
buffer.Add(62)

'add new line
If options & XML_STRIP_NEWLINE = False buffer.Add(10)

Else
'has children need to do opening tag only
'finish opening tag
buffer.Add(62)

'add new line
If options & XML_STRIP_NEWLINE = False buffer.Add(10)

'add children
If children.IsEmpty() = False
For Local child:= Eachin children
child.Export(options, buffer, depth + 1)
Next
Endif

'add value
If value.Length
buffer.Add(value)
If options & XML_STRIP_NEWLINE = False buffer.Add(10)
Endif

'add closing tag
'ident
If options & XML_STRIP_WHITESPACE = False
For index = 0 Until depth
buffer.Add(9)
Next
Endif

'tag
buffer.Add(60)
buffer.Add(47)
buffer.Add(name)
buffer.Add(62)

'add new line
If options & XML_STRIP_NEWLINE = False buffer.Add(10)
Endif
End
')
After the changes above and retrying with the example code I provided, the result will now look like this:
format_code('
<?xml version="1.0" encoding="UTF-8"?>
<test>
<mychild>
I have a value!
</mychild>
</test>
')
I also went ahead and had it check the options for the XML_STRIP_NEWLINE so that it played nicely there too :)


Skn3(Posted 2013) [#95]
Thanks computercoder! Thats very helpful, all I have to do is copy and paste :D

Much appreciation. I also added a few tweaks.

'version 19
' - added GetChildren() override to get ALL children (thanks computercoder)
' - added fixes to Export method, thanks computercoder
' - added GetDescendants) override to get all
' - added result param to all GetChildren/GetDescendants methods. This lets you pass in the list that results will be populated in



computercoder(Posted 2013) [#96]
Glad I could help you :)

Thanks for modifying your source with these changes, they will definitely come in handy!


computercoder(Posted 2013) [#97]
SKN3:

Something I just noticed that was introduced this update in the XMLNode class:
format_code('
'internal
Private


'internal
Private
Method Export:Void(options:Int, buffer:XMLStringBuffer, depth:Int)
')

It may not make any difference to the operation on the code, but the next time you update the source, you may want to exclude that :)


Skn3(Posted 2013) [#98]
thanks, fixed .. no new version number though


AdamRedwoods(Posted 2013) [#99]
hi.

i'm having a problem with the input parsing something like this:
format_code('
<?xml version="1.0"?>
<COLLADA version="1.4.0" xmlns="http://www.collada.org/2005/11/COLLADASchema">
<asset>
<contributor>
<author>Wings3D Collada Exporter</author>
<authoring_tool>Wings3D 1.5.2 Collada Exporter</authoring_tool>
<comments/>
<copyright/>
<source_data/>
</contributor>
<created>2013-12-28T09:08:16</created>
<modified>2013-12-28T09:08:16</modified>
<unit meter="0.01" name="centimeter"/>
<up_axis>Y_UP</up_axis>
</asset>
')
it's incomplete, i can send the full file if you need it.
it seems to be dying right at "<comments/>".


Skn3(Posted 2014) [#100]
Ooo XML BUG!

Have you got latest version?

Out if interest what happens if you put a space? (E.g <comments />)

A runnable example would be good yeah please.


computercoder(Posted 2014) [#101]
I took a quick look at the code, and using the XML AS-IS, the <comments/> doesn't parse. BUT, if you try <comments /> it handles the code correctly (assuming you do the same with each null terminated line like it in the XML).

Here is my test code:
format_codebox('
#TEXT_FILES="*.xml"

Import mojo
Import monkey
Import xml

Function Main()

New testXml

End

Class testXml Extends App

Field XmlLoaded:Bool = False

Method OnCreate()

SetUpdateRate 30

End

Method OnClose()
EndApp
End

Method OnUpdate()

End

Method OnRender()

Cls

If XmlLoaded = False
ReadXml()
XmlLoaded = True
Endif

End

Method ReadXml()

Local XmlError:XMLError = New XMLError
Local RootXmlNode:XMLDoc = ParseXML(LoadString("test.xml"), XmlError)
Local XmlNode:XMLNode = Null

If XmlError.error = False
Local NodeList:List<XMLNode> = RootXmlNode.GetChildren()
For XmlNode = Eachin NodeList
Print XmlNode.name
Next

Else
Print XmlError.ToString()

Endif
End

End
')

Here is the "test.xml" file modified from AdamRedwoods' file:
format_codebox('
<?xml version="1.0" encoding="UTF-8"?>
<collada version="1.4.0" xmlns="http://www.collada.org/2005/11/COLLADASchema">
<asset>
<contributor>
<author>Wings3D Collada Exporter</author>
<authoring_tool>Wings3D 1.5.2 Collada Exporter</authoring_tool>
<comments />
<copyright />
<source_data />
</contributor>
<created>2013-12-28T09:08:16</created>
<modified>2013-12-28T09:08:16</modified>
<unit meter="0.01" name="centimeter"/>
<up_axis>Y_UP</up_axis>
</asset>
</collada>
')


computercoder(Posted 2014) [#102]
@SKN3:

I found and fixed the issue, if you want the behavior I made it use.
Basically, the current code requires a space between the name tag and the "/". My code allows the "/" to be right beside it in the case Adam presented.

Here's what I fixed to ParseXML:

1) Added variable declaration
format_codebox('
Local prevRawAsc:Int = 0
')

2) Added code just below the For rawIndex = 0 Until raw.Length:
format_codebox('
If rawIndex > 0 prevRawAsc = rawAsc
')

3) Added to the code in the If inQuote = False , where you start seeing what is there to parse and need to be selective on what you parse. It is under "Case 47 '/"

Changed this:
format_codebox('
If hasTagName
')

To this:
format_codebox('
If hasTagName Or (prevRawAsc >= 65 And prevRawAsc <= 90) Or (prevRawAsc >= 97 And prevRawAsc <= 122)
')

This fix may need additional characters to check for. The idea was to make sure there was an alpha character, but you may also check numbers or special characters too.

4) Finally, I wrapped the check with the error for "mismatched end tag" as follows:

format_codebox('
If isCloseSelf = False
If parent = Null or tagName <> parent.name
'error
If error error.Set("mismatched end tag", rawLine, rawColumn, rawIndex)
Return Null
EndIf
EndIf
')

Executing this code ran Adam's XML (after I added the closing </COLLADA> tag) without errors :)


AdamRedwoods(Posted 2014) [#103]
nice, thanks!


Skn3(Posted 2014) [#104]
update:
'version 22
' - fixed self closing tags where no space is provided e.g. <tag/> - thanks AdamRedwoods and ComputerCoder
' - moved into jungle solution
' - added test example 5 for testing self closing tags without space


Cheers Adam and ComputerCoder for the bug report. Much thanks for the fix but it was basically a logic error instead of character lookup proposed. The fix was to make sure the correct flags were set on and off at the right time.


computercoder(Posted 2014) [#105]
Yeah, I saw there was a logical error of sort. I was just trying to get something out there that would work. I knew that it needed the flags set right to make it behave correctly, I just wasn't fully sure on how everything was operating (and I was in the middle of sorting out some other issues in my GUI logic as well) so out this came! :) At least it worked! =D However, my code provided has a technical flaw and limitation of the possibility of missing non-alpha characters. Unless its verbose enough to handle ALL of them, it will still have its times at allowing specific instances to pass when they should otherwise be caught.

I'll have a look at how you implemented the fix :)


Skn3(Posted 2014) [#106]
Yeah it took me a while to get my head around it, been a while since I had to look at the actual parser. Basically I added some conditions when it sees a / to see if the name has already been processed. Then a step near the end to turn on a flag inbetween processing the string buffer and processing the tag.

Because the processing of the tag only happens when non alphanumeric character is matched, it meant that it was hitting / and attempting to close the tag that had not been opened yet. Chicken say hello to egg!


computercoder(Posted 2014) [#107]
I was almost there! I had the flag needs setting part and I knew it had the tag name, but I kept thinking "how do I tell the code it has it? And how are these flags being set? guess I'll step through it all.."

Then a step near the end to turn on a flag inbetween processing the string buffer and processing the tag.


I saw it, but the processing tag part completely flew by me... Early morning (between 2 and 3 AM) coding does this I guess? :P

I also took a look at how you implemented it - and as soon as I saw the processing tag part, it was so obvious. I chuckled at my fix =8D


Skn3(Posted 2014) [#108]
I might have to do a bit more detailed flow of code commenting in the future. Just so its easier to come back to :D


Skn3(Posted 2014) [#109]
'version 23
' - added tweak/fix to parser to ignore doctype tag (later on can add support) (cheers copper circle)
' - added tweak/fix to parser to allow : in tag/attribute names (later can add support for contexts) (cheers copper circle)



CopperCircle(Posted 2014) [#110]
Thanks for the updates :)


maltic(Posted 2014) [#111]
Just thought I should let you know I am finding this very useful for parsing Tiled maps and serialized data for my game. Thanks!


computercoder(Posted 2014) [#112]
@maltic

I use this code to parse the XML files created by Tiled. Skn3 did a GREAT job building this module! So whenever I find something that could benefit it, I'm more than happy to pitch in and give up ideas and/or code to him. It's the least I can do for his generosity and time :)


Skn3(Posted 2014) [#113]
:D shucks!

Also thank the army of bug reporters who bug tested my bug riddled code ;)


Difference(Posted 2014) [#114]
'version 23
' - added tweak/fix to parser to ignore doctype tag (later on can add support) (cheers copper circle)



I'm still getting errors on the doctype tag, but only on OS X, for example on this one:

https://github.com/Difference/Monkey-SVG/blob/master/svg.data/tiger.svg


Difference(Posted 2014) [#115]
Putting a space between the doctype tag and the first svg tag fixes it ...


Skn3(Posted 2014) [#116]
Cool, will take a look this afternoon and fix it :D


Goodlookinguy(Posted 2014) [#117]
Maybe it's just me, but this seems to be outputting something quite unexpected.

format_code('Function Main:Int()
Local text:String = "<root><text>Hello <i>W<b>orld</b>!</i></text></root>"
Local xmlErr:XMLError
Local xml:XMLDoc = ParseXML(text, xmlErr)
If xml = Null Error(xmlErr)

Local node:XMLNode
For node = EachIn xml.children
DisplayNodes(node)
Next
End

Function DisplayNodes:Void( node:XMLNode, depth:Int = 0 )
Print node.name + " = " + node.value
depth += 1
For node = EachIn node.children
DisplayNodes(node, depth)
Next
End')

Output
format_code('text = Hello
i = W!
b = orld')

I thought it would take and parse the two sides as separate nodes. It surprised me when it put them together. I was hoping to use your XML parser as a text builder for a text system I was building, but it looks like I'll just manually make something for now.


Difference(Posted 2014) [#118]
@Skn3: When you're in there, you should probably rip out that Print raw[rawIndex..] at line 1845 that keeps spitting out xml in my debug window :-)


V. Lehtinen(Posted 2014) [#119]
Hi! I made a TileD tilemap importer with this great XML module!
It so far only loads all possible data that those tilemaps have - at least I think it does.
Also supports external tilesets (*.tsx).

But please note:
- Code doesn't support zlib/gzip compressed tile data
- Code doesn't support Base64 encoded tile data
- These classes are only data collections so far

If someone still finds this code useful or potential, please feel free to continue or use it. :)

Cheers!


Usage:
format_code('Function Main:Int()
Local tmap:= New TiledMap("path_to_tilemap.tmx")
End')

tiledmap.monkey
format_codebox('

Strict

Import mojo
Import skn3.xml
Import os

Alias LoadString = mojo.app.LoadString

Class Tilemap 'A base class in my game engine, kinda useless here...
Field width:Int, height:Int
Field tileWidth:Int, tileHeight:Int
End

Class TiledMap Extends Tilemap
Private
Field realPath:String
Field doc:XMLDoc, error:XMLError
Public
Field tilesets:= New StringMap<TiledTileset>
Field layers:= New StringMap<TiledLayer>
Field objectGroups:= New StringMap<TiledObjectGroup>


Method New(path:String)
realPath = ExtractDir(RealPath(path))
Self.LoadXML(LoadString(path))
End

Private '// PRIVATE //

Method LoadXML:Void(xml:String)
If xml.Length = 0 Then Return 'note: Proper error

Self.error = New XMLError
Self.doc = ParseXML(xml, Self.error)

If error.error or doc = Null Then Print(error.ToString())
If doc.children = Null Then Error("Unable to load tilemap; no data.")

Local map:XMLNode = doc.GetChild().GetParent()

Local tiledVersion:String = map.GetAttribute("version") ' Not used
Local orientation:String = map.GetAttribute("orientation") ' Not used

width = Int(map.GetAttribute("width"))
height = Int(map.GetAttribute("height"))
tileWidth = Int(map.GetAttribute("tilewidth"))
tileHeight = Int(map.GetAttribute("tileheight"))

Local tileset:TiledTileset, layer:TiledLayer, objGroup:TiledObjectGroup

For Local node:XMLNode = EachIn map.children
Select node.name
Case "tileset"
tileset = LoadTileset(node)
If tileset <> Null Then
tilesets.Set(tileset.name, tileset)
End

Case "layer"
layer = LoadLayer(node)
If layer <> Null Then
layers.Set(layer.name, layer)
End

Case "objectgroup"
objGroup = LoadObjectGroup(node)
If objGroup <> Null Then
objectGroups.Set(objGroup.name, objGroup)
End

End

'Debugging, data-structure-printing
'PrintXMLNode(node)
Next

End

Method LoadTileset:TiledTileset(node:XMLNode)
If node = Null Then Return Null
If node.name <> "tileset" Then Return Null

Local tileset:= New TiledTileset()


' Get tileset properties
Local props:XMLNode = node.GetChild("properties")
If props <> Null Then
tileset.properties.Extend(props)
End

tileset.firstGID = Int(node.GetAttribute("firstgid", "1"))

If node.HasAttribute("name") 'Load tileset
LoadTilesetData(node, tileset)

Else If node.HasAttribute("source") 'Load external tileset
Local extTileset:XMLDoc = ParseXML(LoadString(node.GetAttribute("source")), error)
If error.error or extTileset = Null Then Error("File '" + node.GetAttribute("source") + "' " + error.ToString())

LoadTilesetData(extTileset.GetChild().GetParent(), tileset)
End

Return tileset
End

Method LoadTilesetData:Void(node:XMLNode, tileset:TiledTileset)
If node = Null or tileset = Null Then Return

tileset.name = node.GetAttribute("name")
tileset.tileWidth = Int(node.GetAttribute("tilewidth"))
tileset.tileHeight = Int(node.GetAttribute("tileheight"))

Local imgNode:XMLNode = node.GetChild("image")
If imgNode = Null Then Error("Tileset '" + node.GetAttribute("name") + "' has no image?")

' Get properties for individual tiles
Local tileNode:XMLNode, prop:TiledPropertySet, tempProps:XMLNode
For tileNode = EachIn node.children
Select tileNode.name
Case "tile"
tempProps = tileNode.GetChild("properties")
prop = New TiledPropertySet()
prop.Extend(tempProps)

Local id:Int = Int(tileNode.GetAttribute("id"))
tileset.tileProperties.Set(id, prop)
End
Next
End

Method LoadLayer:TiledLayer(node:XMLNode)
If node = Null Then Return Null
If node.name <> "layer" Then Return Null

Local layer:= New TiledLayer()

layer.name = node.GetAttribute("name")
layer.width = Int(node.GetAttribute("width"))
layer.height = Int(node.GetAttribute("height"))

' Get layer properties
Local props:XMLNode = node.GetChild("properties")
If props <> Null Then
layer.properties.Extend(props)
End

LoadLayerData(node, layer)

Return layer
End

Method LoadLayerData:Void(node:XMLNode, layer:TiledLayer)
Local data:XMLNode = node.GetChild("data")
If data = Null Then Print("WARNING: Layer '" + layer.name + "' has no data!")

Local tileNode:XMLNode, x:Int = 0, y:Int = 0
For tileNode = EachIn data.children
If x >= width Then
y += 1
x = 0
End

If tileNode.name = "tile" And tileNode.HasAttribute("gid") Then
layer.SetTileID(x, y, Int(tileNode.GetAttribute("gid")))
End

x += 1
Next
End


Method LoadObjectGroup:TiledObjectGroup(node:XMLNode)
If node = Null Then Return Null
If node.name <> "objectgroup" Then Return Null

Local objGroup:= New TiledObjectGroup()

objGroup.name = node.GetAttribute("name")
objGroup.width = Int(node.GetAttribute("width"))
objGroup.height = Int(node.GetAttribute("height"))

' Get group properties
Local props:XMLNode = node.GetChild("properties")
If props <> Null Then
objGroup.properties.Extend(props)
End

' Load objects from group
Local child:XMLNode, obj:TiledObject
For child = EachIn node.children
If node.name = "object" Then
obj = LoadObject(child)
If obj <> Null Then objGroup.objects.AddLast(obj)
End
Next

Return objGroup

End

Method LoadObject:TiledObject(node:XMLNode)
If node = Null Then Return Null

Local obj:= New TiledObject

obj.name = node.GetAttribute("name")
obj.type = node.GetAttribute("type")
obj.x = Int(node.GetAttribute("x"))
obj.y = Int(node.GetAttribute("y"))
obj.width = Int(node.GetAttribute("width", "0"))
obj.height = Int(node.GetAttribute("height", "0"))

' Get object properties
Local props:XMLNode = node.GetChild("properties")
If props <> Null Then
obj.properties.Extend(props)
End

Return obj
End

Public '// PUBLIC //


End

Class TiledTileset
Field name:String
Field image:Image
Field width:Int, height:Int
Field widthInTiles:Int, heightInTiles:Int
Field tileWidth:Int, tileHeight:Int
Field firstGID:Int = 1
Field properties:= New TiledPropertySet
Field tileProperties:= New IntMap<TiledPropertySet>

End

Class TiledLayer
Field name:String
Field width:Int, height:Int
Field properties:= New TiledPropertySet
Field tiles:= New IntMap<Int>

Method SetTileID:Void(x:Int, y:Int, id:Int)
If width = 0 Then Return
tiles.Set(y * width + x, id)
End

Method GetTileID:Int(x:Int, y:Int)
If width = 0 Then Return - 1
tiles.Get(y * width + x)
End

End

Class TiledObjectGroup
Field name:String
Field width:Int, height:Int
Field properties:= New TiledPropertySet
Field objects:= New List<TiledObject>

End

Class TiledObject
Field name:String, type:String
Field x:Float, y:Float, width:Int, height:Int
Field properties:= New TiledPropertySet

End

Class TiledPropertySet

Method Get:String(name:String)
Return keys.Get(name)
End

Method Get:Int(name:String)
Return Int(keys.Get(name))
End

Method Get:Float(name:String)
Return Float(keys.Get(name))
End

Method Get:Bool(name:String)
If keys.Get(name).ToLower() = "true" or keys.Get(name) = "1"
Return True
Else If keys.Get(name).ToLower() = "false" or keys.Get(name) = "0"
Return False
End
End

Method Contains:Bool(name:String)
Return keys.Contains(name)
End

Method Extend:Void(node:XMLNode)
If node.name <> "properties" Return

For Local child:XMLNode = EachIn node.children
keys.Set(child.GetAttribute("name"), child.GetAttribute("value"))
Next
End

Private
Field keys:= New StringMap<String>
End

Function PrintXMLNode:Void(node:XMLNode, depth:Int = 0)
If node = Null Then Return
If node.name = "tile" Then Return

Print RSet(node.name + " {", depth)
For Local s:String = EachIn node.attributes.Keys()
Print RSet(s + ": " + node.GetAttribute(s), depth + 1)
Next

If node.HasChildren() Then
For Local nod:XMLNode = EachIn node.children
PrintXMLNode(nod, depth + 1)
Next
End

Print RSet("}", depth)
End

Function RSet:String(in:String, depth:Int)
Local spaces:String = ""

For Local i:Int = 0 To depth - 1
spaces += " "
Next

Return spaces + in
End')


Shinkiro1(Posted 2014) [#120]
First, this is an excellent module.

How can I add a newline character in my xml, so that in monkey there is ~n character in the string.
I have already tried ~n, \n and used a newline in my file (by pressing enter ^^).

PS: Basically I have a description which is multiple lines long.


Difference(Posted 2014) [#121]
I just use * for newlines, and substitute it after loading.


Shinkiro1(Posted 2014) [#122]
Thanks Difference, I will go with that workaround for now.
I am still interested if this i built in, as I would expect.


Skn3(Posted 2014) [#123]
As it is XML it doesn't have built in newline escaping. That is upto the client/app to implement. But, you can just hit "enter" in your XML file to start a new line and that *should* appear in your resulting value as a newline.

Otherwise you would need to write something that parses the \n from your text values.


Difference(Posted 2014) [#124]
Hello Skn3 :-) Did you get a chance to look at the doctype bug ?


Skn3(Posted 2014) [#125]
Hullo,

I thought id ressolve some of teh outstanding issues/bugs.

'version 24
' - fixed doctype bug (thanks difference, sorry the delay ;)
' - removed left in print satement (thanks difference, sorry the delay ;)
' - added support for text/whitespace characters (the value of a node) to be split into child nodes. This should be transparently working and you can still use node.value
' - added node.text bool to indicate if the node is a text node
' - added text boolean flag to many of the methods. This allows text nodes to be scanned/returned. The text boolean defaults to false, which will ignore text nodes. For example GetChild(true) would return the first child node, GetChild(false) would find the first NON-text node.
' - added AddText() method this will either add a child node or append to teh nodes value (depending on the parse mode used)
' - added example7.monkey demonstrating text nodes


This is a fairly "big" update and includes functionality Goodlookinguy was requesting a while back.

The following xml file:
format_code('<root>
text in root
<div>Hello
<i>
W
<b>orld</b>
!
</i>
</div>
!!! :D
</root>')

Produces the following output:
format_code('without text nodes:
(node) root (value=text in root!!! :D)
- (node) div (value=Hello)
-- (node) i (value=W!)
--- (node) b (value=orld)


with text nodes:
(node) root (value=text in root!!! :D)
- (text) text in root
- (node) div (value=Hello)
-- (text) Hello
-- (node) i (value=W!)
--- (text) W
--- (node) b (value=orld)
---- (text) orld
--- (text) !
- (text) !!! :D')


Skn3(Posted 2014) [#126]
Another quick update:
'version 25
' - added so newlines in XML will be included in xml text/values
' - added so XML_STRIP_NEWLINE can now be used in ParseXML to strip any newlines within text/value



Kaltern(Posted 2014) [#127]
Hi, I'm trying to parse a fontMetrics XML, but I just can't seem to figure out how to get the values for each character.

The XML basically looks like this:

format_code('
<?xml version="1.0" encoding="utf-8"?>
<fontMetrics file="SmallRedDigi.png">
<character key="32">
<x>8</x>
<y>8</y>
<width>36</width>
<height>60</height>
</character>
<character key="33">
<x>52</x>
<y>8</y>
<width>30</width>
<height>60</height>
</character>
')

I can't seem to get more than the 'key' value returned tho - how can I get each character returned please?


Skn3(Posted 2014) [#128]
Post your code here that you have now and I can show you what to do from there.


Skn3(Posted 2014) [#129]
'version 26
' - fixed typo in missing variable



Skn3(Posted 2015) [#130]
'version 27
' - small mem ref error when removing path list node
'version 28
' - small tweak so that if a node has no children but a value e.g. <value>SomeText</value> it will be formatted onto a single line.



Skn3(Posted 2015) [#131]
'version 29
' - xml nodes now keep their original casing, this has changed the behaviour of node.name, it will return the unmodified name



Skn3(Posted 2015) [#132]
Flurry of XML updates, I am utilising xml doc creation properly for the first time so a few issues are popping up.

'version 31
' - casing now remains in provided format for node attributes
'version 30
' - fixed long standing (but apparently no one caught??) errors with setting/getting value of xml node



Skn3(Posted 2015) [#133]
'version 32
' - improved performance by storing list node pointers when adding/removing xml nodes
' - made it so node.value and node.value = works for text type nodes. Will rebuild parent text value if needed
' - fixed ClearText as it was not properly updating the node pointers so text remained



Difference(Posted 2015) [#134]
Thanks for keeping this essential module alive.


Skn3(Posted 2015) [#135]
No problem, glad people get some usage out of it! Just doing my bit to try and make Monkey more attractive to newcomers!

'version 33
' - recent changes had broken self contained tags on export



Skn3(Posted 2015) [#136]
moooooaarrrr
'version 34
' - reworked internal node code so node.Free() will fully remove self from parent
' - added Remove() so a node can be removed but not freed!



Skn3(Posted 2015) [#137]
small update

'version 35
' - added .AddChild(node) method to allow copying a node object into a parent.



Skn3(Posted 2015) [#138]
'version 36
' - AddChild(node) now has second param (defaults to true) to handle recursing into child nodes



Skn3(Posted 2015) [#139]
'version 37
' - added node.MergeAttributes(node) to let us merge attributes from anotehr node
' - added node.GetAttributes() to fetch all attributes in a stringmap




Bob(Posted 2015) [#140]
Great module! Does it have the ability to actually write the data to an XML file? I'm able to read from my XML file using Local doc:= ParseXML(LoadString("data.xml")). I can then change the data I want and if I call print doc.Export(XML_STRIP_CLOSING_TAGS) (Like in example2.monkey) I can see the the values have been updated. I then use the same Export command (minus "print" of course), but when I open the XML file again, the values are what they were originally. Does Export() not write back to the file or am I using it incorrectly?


Skn3(Posted 2015) [#141]
You should use monkey's file routines to save it out. Export only provides a string as return.


Skn3(Posted 2015) [#142]
'version 38
' - fixed bug in parsing attributes without quotes, preceeding > caused xml error



Samah(Posted 2015) [#143]
Just a note that the latest version of the Diddy Tile engine supports Skn3's XML module as well as the Diddy one.


Skn3(Posted 2015) [#144]
cool, thanks for the support samah :D