Friday 15 January 2016

How to create your own Marker Pack

I would like GW2 TacO to be useful for many people. It was easy to use the tech I built to help out in Spirit Vale, but placing a few directional markers and setting up some locational timers isn't too much work, and the raid audience is limited. This system can be used for so much more, and with the latest build I aim to have it open enough for anyone to be able to create and share markers in an organized fashion.

So here's a reference on how GW2 TacO supports this effort.

What is a marker?

On the face of it, what GW2 TacO does is pretty simple: it asks GW2 what your current location and the camera position are, and draws billboards in 3D space based on that. The trick is knowing where in space to place those points in the first place. The locations come from the information that we already have: your character position. You press a key, the system takes your current position and creates a marker. But if this was all there is to it, there would only be one type of marker and soon enough chaos would ensue. 
The current default marker icon
So besides the position in 3D space, markers need other properties:
  • What image should be drawn for the marker?
  • What size should the marker be in the game world?
  • How transparent should the marker be?
  • At which distance should the marker show up at all?
  • How high off the ground should the marker be? (Your location is actually on the ground)
  • What should happen if you press the action button when you're at the marker?
  • ... and a lot of things I probably haven't encountered yet
Each marker can store all of the information above, and if you wanted you could edit the poidata.xml right now by hand and add these properties to all the markers individually (given that you know the format, but more on that later). So there is a need to somehow organize markers into groups, optimally in a way that lets us keep the option open to change markers individually.

Enter Marker Categories.

An introduction to Marker Categories

A marker category is a description of a group of markers containing some or all of the information mentioned above in the list. Each marker can have a type field that describes which category it fits into. Any values defined by the category will be set for the marker, unless the marker defines them for itself as well - in this case the value coming from the category is overwritten by the value specified in the marker itself. With this we have a system where we both have grouped data (if I change the image for the category all the markers in that category will change their icon) but with the option of having individual markers a say in how they behave (a marker can still have its own separate icon defined with all the rest of the values coming from the category it is in).

Each marker can belong to exactly one category.

But categories can be organized into a hierarchy.
Example hierarchy for categories: resourcenode.chest.mapspecific.verdantbrink
The point here is that categories themselves will often contain very similar data. Think about all the in-world collectible achievements of late: golden lost badges, lost coins, exalted masks, etc. All of these can easily be marked, and their behavior has to be pretty much the same: if you press 'F' on one of these markers, it should disappear forever because you already have the achievement point for that one and it's irrelevant from that point on. They might all however have different icons. Hierarchic categories solve this in the following way: you can define a category (let's say 'collectible') that defines their behavior: 'disappear forever when activated' (more on how to do this later). Then you define categories under 'collectible' for the examples above: 'goldenlostbadge', 'lostcoin', 'exaltedmask'. These categories will inherit all settings from their parents (in this case their behavior), and you can set icons separately for each of them.

Now to set the category for a marker you need to write out the full name of the category, which is the parent-to-child order combination of the names of the categories separated by dots. For example: 'collectible.goldenlostbadge' will refer to the golden lost badge category. Category names are case insensitive to avoid any confusion.

This is all pretty standard, but the biggest point of this system is that it's expandable. Nothing is hardcoded into the overlay and if you distribute your own marker pack that can have its own category definitions that will seamlessly integrate into the hierarchic system. The final category tree you will see in the overlay is a combination of all the categories loaded from all the marker files.

Data formats, file system info, what goes where

GW2 TacO stores all data in xml files. There a bunch of these, but the ones used in displaying markers are the following:

  • poidata.xml - this is your local marker database, it contains all the markers you placed on your own. It can also contain category descriptions.
  • categorydata.xml - this file is loaded first, it contains a default category tree. This category tree for now is pretty much only preliminary and will be expanded/rearranged later.
  • activationdata.xml - this file contains when you last pressed 'F' on a marker that has a behavior different than 'always visible'.
  • any xmls under the POIs directory will be loaded. The format of these matches the format of poidata.xml, however the markers in these data files will always be loaded. If you get a new marker pack you just need to drop it in there and restart the overlay.
All of these xml files have the root tag <OverlayData>. All tags in the xml files are case sensitive.

Marker description


Markers in the xml files are described under the <POIs> tag under the root <OverlayData> tag. The <POI> tag describes a marker. The following attributes can be set for each marker:
  • MapID - integer, required. Describes which map the marker is located on.
  • xpos - float, required. X position of the marker in world space.
  • ypos - float, required. Y position of the marker in world space.
  • zpos - float, required. Z position of the marker in world space.
  • type - string, optional. This is the full name of the category the marker belongs to. If this is not set, the marker will be presented with default parameters. Must not contain any spaces or special characters.
  • GUID - base64 encoded string, optional. This is a unique identifier for the marker used in tracking activation of markers through the activationdata.xml file. If this doesn't exist for a marker, one will be generated automatically and added on the next export.
  • The following tags are all optional, and can be both set for each category (description below) and each marker individually. Ideally these should be set in the categories and only overridden for a handful of icons if at all, but the option is there.
    • iconFile - string. The icon to be displayed for the marker. If not given, this defaults to the image shown at the start of this article. This should point to a .png file. The overlay looks for the image files both starting from the root directory and the POIs directory for convenience. Make sure you don't use too high resolution (above 128x128) images because the texture atlas used for these is limited in size and it's a needless waste of resources to fill it quickly.
    • iconSize - float. The size of the icon in the game world. Default is 1.0 if this is not defined. Note that the "screen edges herd icons" option will limit the size of the displayed images for technical reasons.
    • alpha - float. How opaque the displayed icon should be. The default is 1.0
    • behavior - integer. This is an important one, it describes the way the marker will behave when a player presses 'F' over it. The following values are valid for this parameter:
      • 0: the default value. Marker is always visible.
      • 1: 'Reappear on map change' - this is not implemented yet, it will be useful for markers that need to reappear if the player changes the map instance.
      • 2: 'Reappear on daily reset' - these markers disappear if the player presses 'F' over them, and reappear at the daily reset. These were used for the orphan markers during wintersday.
      • 3: 'Only visible before activation' - these markers disappear forever once the player pressed 'F' over them. Useful for collection style markers like golden lost badges, etc.
      • 4: 'Reappear after timer' - This behavior makes the marker reappear after a fix amount of time given in 'resetLength'.
      • 5: 'Reappear on map reset' - not implemented yet. This will make the marker reappear when the map cycles. In this case 'resetLength' will define the map cycle length in seconds, and 'resetOffset' will define when the first map cycle of the day begins after the daily reset, in seconds.
      • 6: 'Once per instance' - these markers disappear when triggered but reappear if you go into another instance of the map
      • 7: 'Once daily per character' - these markers disappear when triggered, but reappear with the daily reset, and can be triggered separately for every character
    • fadeNear - float. Determines how far the marker will start to fade out. If below 0, the marker won't disappear at any distance. Default is -1. This value is in game units (inches).
    • fadeFar - float. Determines how far the marker will completely disappear. If below 0, the marker won't disappear at any distance. Default is -1. FadeFar needs to be higher than fadeNear for sane results. This value is in game units (inches).
    • heightOffset - float. Specifies how high above the ground the marker is displayed. Default value is 1.5
    • resetLength - integer. For behavior 4 this tells how long the marker should be invisible after pressing 'F'. For behavior 5 this will tell how long a map cycle is.
    • resetOffset - integer. Not currently used, this will supply data for behavior 5. The data will be given in seconds.
    • DisplayName - string. This text is used to display the type of the marker. It can contain spaces.
    • color - hex value. The color tint of the marker
    • autoTrigger - boolean. Determines if going near the marker triggers it
    • hasCountdown - boolean. Determines if a marker has a countdown timer display when triggered
    • triggerRange - float. Determines the range from where the marker is triggered
    • minSize - integer. Determines the minimum size of a marker on the screen, in pixels.
    • maxSize - integer. Determines the maximum size of a marker on the screen, in pixels.
    • achievementId - integer. An ID for an achievement from the GW2 API. Markers with the corresponding achievement ID will be hidden if the ID is marked as "done" for the API key that's entered in TacO. For a list of achievement IDs see https://wiki.guildwars2.com/wiki/API:2/achievements
    • achievementBit - integer. This is similar to achievementId, but works for partially completed achievements as well, if the achievement has "bits", they can be individually referenced with this.
    • info - string. This can be a multiline string, it will show up on screen as a text when the player is inside of infoRange of the marker
    • infoRange - float. This determines how far away from the marker the info string will be visible
    • toggledefault - boolean. If set, the category will be enabled/disabled by default when loading the marker pack (0 = disabled)
    • festival - string. A comma separated list of strings (without spaces) from the following pool that makes the markers in this category only show up during the associated festival: halloween, wintersday, superadventurefestival, lunarnewyear, festivalofthefourwinds, dragonbash
    • copy - string. This string will be copied to the clipboard when the marker is triggered. Usually used with autotrigger enabled.
    • copy-message - string. This string will be displayed in the middle of the screen when the clipboard copy is triggered. Useful to let the user know that the clipboard has been updated.
    Example: 

    <OverlayData>
      <POIs>
        <POI MapID="1062" type="collectible.journalentry" xpos="-266.477" ypos="52.4114" zpos="-628.789" GUID="XtFDXUUJO0aDsDzjZOt7NQ=="/>
      </POIs>
    </OverlayData>

    Category description


    Categories in the xml files are described by <MarkerCategory> tags under the root <OverlayData> tag. They can be nested to build a hierarchy. Any marker parameter defined in a category will propagate down to categories nested in it unless they opt to override the parameter. Besides the marker parameters described above the only thing that can be set for a category is the name parameter. 

    Example:

    <OverlayData>
      <MarkerCategory name="Collectible">
        <MarkerCategory name="JournalEntry" iconFile="Data\raid1_achievements.png" behavior="3"/>
      </MarkerCategory>  
    </OverlayData>

    If you're planning on creating a marker pack and your markers don't fit into existing categories I suggest you start with an empty poidata.xml and create your categories in that file. This way if you distribute your poidata.xml people can drop it into their POIs directory and load your markers easily.

    Creating categories


    For the time being you'll need to create new categories by hand using any text editor. Just add the category to your poidata.xml, restart TacO and you're good to go.

    Trails


    Trails are a feature meant to optimize the creation of routes and to remove marker clutter from the system. A trail is a continuous line that is recorded in a recording session by simply walking around. They are a bit different from markers in that the trail data needs to be exported from the recording session in order to be usable, and it's stored in a dense binary format that's not easily editable by hand.

    How to record a trail


    To record a trail, you need to have the Marker Editor window open.
    You can start recording a trail simply by pressing the Start New Trail button. Everywhere you walk on the map, the trail will be built. You can pause recording with the Pause Recording button, which will simply stop the accumulation of trail points behind you while the pause function is enabled. This should allow you to inspect the results of a recording session before exporting it.

    The Delete Last Segment button allows for a rudimentary type of undo. It should come in handy if you miss a jump for example and don't want to redo the whole trail up to that point. It's highly recommended to bind this functionality to a key.

    Once your trail is finished, you can export it with the help of the Export Trail button.

    Stopping a recording session will also restart the trail, so be careful to always export a finished trail before starting to record a new one.

    Using and sharing trails


    In order to get an exported trail to show up in TacO, it needs to be added to a marker pack just like a marker. Trails are described by the <Trail> tag and uses the same category system as the markers. If you put a marker and a trail in the same category, the user can hide them both at the same time by hiding the category.
    Here's an example trail:

    <OverlayData>
    <POIs>
    <Trail trailData="Trails/poobadoo.trl" texture="data/Bounty.png" color="ffffffff" animSpeed="1" alpha="1" type="tactical.guildmission.bounty.poobadoo" fadeNear="3000" fadeFar="4000"/>
    </POIs>
    </OverlayData>

    The color, type, alphafadeNear and fadeFar attributes function the same as they do for markers.
    The trailData tag needs to point to a binary trail. These are the files that you get by exporting them during a recording session. The binary trails also contain the map they were recorded on, so the MapID tag is ignored for trails.
    The texture tag points to the texture that should scroll on the trail.
    The animSpeed tag is a float value that modifies the speed of the animation on a trail.
    There's also a trailScale tag that is a float value that modifies how stretched the texture will look on the trail.

    In order to share trails you need to include the binary files into the marker packs just like you do with images.

    Placing your own Markers

    This is the easy part. Any markers you place will be added into poidata.xml. You can place markers by default using the '+' button and remove them with the '-' button.
    If you have the marker editor open (toggle it open in the TacO menu, then turn on window edit mode to move and resize the window to the correct position) and you're standing on a marker you placed, the marker editor will show the type of the marker you're standing on, a button to change that category, the default type a new marker gets assigned to and a button to change the default category. Assuming you want to create a marker pack for similar items all you need to do is select a default and visit all the locations you want marked. The poidata.xml will be written every time you add or remove a marker.
    To override category settings for individual markers you'll need to edit the xml by hand for the time being. Later updates will address this issue.

    Ideal targets for marking

    • world chest locations
    • any type of location based achievements like lost coins etc (for these I suggest you use the corresponding achievement icon)
    • map chest spawn positions (think dry top, silverwastes, HoT maps, etc)
    • JP start locations
    • where to place siege in WvW
    • etc
    If you create a high quality (all positions marked, precisely) marker pack send it my way and I'll put it in the next release.

    Sharing

    If you have created a marker pack that you feel can be shared with the community simply rename the poidata.xml, gather the images you used, zip them all up, and tell people to decompress the package into the POIs directory next to TacO. It's also wise to try the marker pack on a freshly uncompressed download of TacO this way to make sure that the markers load the correct icons.
    That's it!

    What's still missing

    As you might have noted, some features are still under development. Anything that says "not implemented yet" above will be coming soon.

    55 comments:

    1. Really excited for future updates and thank you for the in depth explanations.

      Not sure if anyone else has been working on Silverwastes chests but I'm pretty close to completing marking all the chests.

      ReplyDelete
      Replies
      1. Still working on the Silverwastes chests, I have most of them complete from running chest trains but there's a few odd ones I need to get a few shovels for to check them off.

        In the meantime though my guild and I put together a group of like ten mesmers and did the golden lost badges "Go for the Gold" achievement in silverwastes. I'm putting a link with the zip below. I tried to follow your instructions to the T - downloaded a fresh installation and put the xml and icon file into the POIs folder and everything worked perfectly so let me know if there's any problems. Also not sure if you want to use the official golden lost badge icon or not, they're a little difficult to see.

        https://www.dropbox.com/s/18y0xd3hv4ciesj/goldenlostbadges_pois.zip?dl=0

        Delete
      2. Wow, you really started with the hardest one I see :) I'll put this in the next release if that's all right with you, but I will replace the icon to this one: https://wiki.guildwars2.com/wiki/File:The_Silverwastes.png

        Delete
    2. Ha, yes. I had started the golden lost badges in the past and had a lot of trouble with them. I was lucky to get a bunch of people to help me out this time though.

      Looks great. I'll pass anything else I get off to you in the near future.

      ReplyDelete
    3. what unit for fadeFar & fadeNear

      im sure it not skill range unit

      ReplyDelete
      Replies
      1. It used to be world units (which is in meters), the latest release changes it to skill range units (inches), that should make it easier to set these.

        Delete
    4. hope can hide icon when zoom in and it near, cos icon will block screen

      ReplyDelete
    5. I created a .xml file for marker set graphics for jumping puzzle guides which worked fine while the POI data was in poidata.xml in the same directory as the .exe, but when I moved the jump puzzle POI to a separate .xml file in the POI directory the markers all reverted to the default marker graphic. When I moved them back to the poidata.xml file they found the .png files again.

      ReplyDelete
      Replies
      1. Please send me all the data needed so I can debug this. I'll only have time for it next week though.

        Delete
      2. I found the cause of my problem. I extracted the POIs from poidata.xml and put them in an .XML file in the POIs directory. There were no Marker Categories defined in that file, as I have all of my marker categories currently in a single file. I changed the POIs-only filename so it starts with "POI". Now the POIs show as they did in their original location.

        Delete
      3. Hey Phil, are you still working on the JP guide?
        If so, please tell me, because while I manage to get all possible Nodes I also decided to add all permanent chests.
        But if you still work on the guide I guess you will add the chests of the JPs by yourself, right? :)

        Delete
    6. How to make a poi like speed brost at vg?

      ReplyDelete
    7. Hi,
      TacO is a great job !!!
      Actuelly, i'm planing create a route to all JP.
      Looking for your documentation, i don't see the prupose of TriggerRange fonction in POI. What is it for ?

      ReplyDelete
    8. I've started to make a resourcenode list, planning one Map per day.
      So I wanted to ask if there is already a list?
      Because if there is one it would be a waste of time to make a new one :P

      ReplyDelete
      Replies
      1. No resource node list yet. Do note that not all nodes spawn at all locations in all instances so it might be a huge task...

        Delete
      2. Yes I know, I walked many times throught Mount Maelstrom today and found new Nodes everytime.
        I will make one List with all Maps as precise as I can so that it's useable and then update it from time to time to get nearly all of them.

        The HoT-maps have the same problem. They are useful if you search the chests but there are also some chests which aren't in the list yet ^^

        I'll send you a mail in around 3-4 weeks when I think I've got them all :D

        Delete
      3. Oh and there is a little bug with the marker, I guess.
        After setting up around 15 marks I have to press minus some times to be able to set up new marks.

        Delete
      4. That means there's a marker there already. Make sure you don't have double placed markers as that'd be problematic to check the complete pack for.

        Delete
      5. Ok I'm actual rechecking all nodes to get sure there is no doubled one.
        Could you pls add additional entries for Bloodstone Crystals and Petrified Stumps in a next update?

        Delete
      6. You can do that on your own. It's described in this very article ;)

        Delete
    9. It's now one week and I managed to "finish":
      Queensdale
      Timberline Falls
      Mount Maelstrom
      Southsun Cove
      Dragons Stand
      Bloodstone Fen
      Ember Bay

      While moving around Ember Bay I detected a small bug which made my TacO crash on exit.
      There were spaces inside of collectibles_mursaattokens.xml and collectibles_tabletsoftheunseen.xml. I removed them and everthing worked fine again.

      I tried to use fadeNear and fadeFar because of the mass of icons floating around at the moment but it seems to not work properly yet. I've seen the entries inside of the mursaattokens.xml and haven't seen any fading icon on Ember Bay.

      I hope Phil B will answer me soon because I have a friend who is interested to do the thing he wanted to do :P
      Or do you have any information about the JP guide?

      ReplyDelete
      Replies
      1. new Maps "finished":

        Dry Top
        Silverwastes
        Tangled Depths

        Delete
    10. This comment has been removed by the author.

      ReplyDelete
    11. This comment has been removed by the author.

      ReplyDelete
      Replies
      1. There's no near fadeout functionality. FadeNear defines the distance where the marker starts to fade out and FadeFar defines the distance at which it's completely invisible.

        Delete
    12. Hi, and first of all, thanks for this great tool.
      I'm just having a problem with fadeNear : the icon doesn't disappear when I approach, whatever value I set. fadeFar works as expected.
      Here is my POI data : POI MapID="54" xpos="-746.345" ypos="35.1081" zpos="505.179" GUID="Ztnc+hMy806A81UUw/8ZkA==" fadeNear="1800.0" fadeFar="3600.0"/
      (Had to remove inf and sup signs to display data)

      ReplyDelete
      Replies
      1. See reply above:
        There's no near fadeout functionality. FadeNear defines the distance where the marker starts to fade out and FadeFar defines the distance at which it's completely invisible.

        Delete
      2. ok, understood now ^^, thanks for your fast awnser ;)

        Delete
    13. Small thing, sorry to bother you about a typo and not sure the best way to contact you!

      It says "Filter Displayed Tactical Makers" rather than Markers in your application.

      ReplyDelete
    14. This comment has been removed by the author.

      ReplyDelete
    15. thank you very much. the latest update gives you a completly new oportunity to create helpful informations for others.
      cant wait for the next update.

      ReplyDelete
    16. Hi, is there a way to make changing the categories of downloaded markers? Each marker creator makes different categories and they become a mess to find. Would be nice if we could move them to subfolders (categories) of our choices once installed so we find each marker easily in the taco menu.

      ReplyDelete
    17. Maybe I have the dumb, but I don't see an export trail button. What am I missing?

      ReplyDelete
      Replies
      1. Do you have the marker editor open and resized so it shows every button?

        Delete
      2. Yeah my marker editor is blank. Maybe I'll get a fresh install or sumthin.

        Delete
      3. i have to create a marker at either end. got it now.

        Delete
    18. Is there a way to reload poi markers without reloading Taco itself?

      ReplyDelete
      Replies
      1. It's already on the todo.

        Delete
      2. Ah, haven't noticed, thank you!
        I have second question tho: Are there any plans for MapID to be inherited?
        For example if I'm creating map specific marker pack, I would like to define MapID in MarkerCategory and don't include it in tags so the code would stay clear and short.

        Delete
      3. The idea is that markers contain the complete location information for clarity's sake, which is why I didn't do this.

        Delete
    19. I may be missing something simple but how do you change the trail markers? I can't seem to figure it out.

      ReplyDelete
    20. You can load a stored trail and continue through it and save it again, that's the only way to "change" a trail, since it's stored in a binary format there's no easily hand editable file.

      ReplyDelete
    21. when i turn on "hasCountdown" it automaticly turns on "autotrigger", is it supposed to do that? its really a shame that you cant have the cooldown timer on pressing F...............................

      ReplyDelete
    22. Some of the Firestone Shards in Draconis Mons are automatically picked up when you get close to them (as they are stuck to a wall and you just glide past them) - there's the autoTrigger property I can set but ="1" and ="True" don't seem to do anything. Am I not using it right, or is that flag only for countdowns?

      ReplyDelete
      Replies
      1. What's the question? Do you want to replicate their functionality or stop them from triggering automatically?

        Delete
      2. I want the markers to trigger automatically when gliding or flying near them as that's how the Firestone Shards are picked up - as you don't need to press F to pick up the Shard, the mark should ideally be triggered automatically.

        The alternative is to give it a large trigger range so that although you still have to press F to clear the marker, there's a large margin of error.

        Delete
      3. The automatic trigger still uses the trigger range. Just check the Firestone Shard marker packs to see their settings to see how it's done.

        Delete
      4. I couldn't find one to check, Google turns up nothing. I see now it's not indexed anywhere, just gotta download marker packs until one contains it =/

        Thanks =)

        Delete
      5. If it shows up in TacO, it's in an xml file in your folder somewhere ;)

        Delete
    23. i wish there would be trails for HoT maps completion (even with mounts) so i can finally do it easily... Well, no i don't wish... i'd kill for it :)

      ReplyDelete
      Replies
      1. I think for complete Maps it is a good idea to check for a guide on dulfy(dot)net. She usually has pretty good content.

        Delete
    24. What format is the color in? It's 8bit and never come across that in programming. 6bit yes. RRGGBB

      ReplyDelete
    25. https://i24.servimg.com/u/f24/12/61/11/36/screen24.jpg
      Hello,
      given the screen that this gives, would it be possible to have a GLOBAL selection of display of a maximum distance, because it is obvious that more than 500 meters I do not care!

      ReplyDelete