October 2016

October 3

Plowing right along, now that the delicate things are done with! Mostly anyway...

The result stats tab is now fully setup, except with reserved spots for sections such as handling gear and buffs, where those tabs don't even exist/function yet. With this, the v3 Armory is now almost as functional as the initial release three and a half years ago .

I've begun work on the URI generation as mentioned before, and have ended up making some adjustments based on things I have learned/worked out since that Dev Log.

The previous Set Links were unwieldy for numerous reasons, not least of which being their length. But it was also a problem that they contain characters that are not really supposed to be in URI. Pasting unshortened links fails since these symbols confuse web pages/programs about where the link ends, and aggravates some browsers due to the symbols not being encoded (which is why I had to ditch using the address bar to begin with).

The v3 Set Links will use only "safe" characters, and are still targeted to be significantly shorter than the old links, even with the extra data that they now have to carry. To jump right into it, I'll just paste the section of the Set Link that v3 can successfully generate so far, and then go over how it's made/functions piece by piece:

/Armory/#00_300~01_A.2.1s.Z8.1i.D.0.0.uy.jK.jP.iU.ik.20z.2M.0.1C.2X.Z8.uy.0.0

The /Armory/ bit is just to show what the full URI will roughly look like. The first thing to note is that the v3 URI continue to take the shortcut of using hash values instead of actually being proper URI. Given that this works, I don't have any realistic reason to change it at this point. However, everything past the hash looks completely alien, compared to the last URI, which looks like this for the "same" section of data:

/Armory/#(`version`+2.03,`stats`+[7885,172,1183,1205,1105,1194,90]

The key difference in the structure is that I have abandoned JSON for the purposes of the URI. It was very convenient for the process of saving/loading, but not so convenient in any other category. The new format is pretty much a series of blocks of data, that are in turn composed of other blocks of encoded data. These blocks are numbered (I will explain this below), but like with the old Set Links, the order the blocks are in does not actually matter, just like with the JSON data.

The "outermost" layer of blocks are these:
00_ = Version Data
01_ = Base Stat Data
02_ = Skill Rank Data
03_ = Transformation Path Data
04_ = Equipment Data
05_ = NX Equipment Data
06_ = Buff Data

Each block is identified via those prefixes, and the blocks are separated with the ~ character. When read, the read function can simply perform .split("~") to isolate these blocks.

00_ Version Data:
00_300

Version data is recorded for the purposes of identifying the overall structure of the URI in the event of a major shift or a change that otherwise would break all old URI. It checks this value to know which format to read it in. Version is stored as a series of three Base 62 digits. The first digit is the overall version number, 3 in this case. The second digit is the update number, and the third digit is the patch number for the update. Overall, this results in Version 3.00.00 being saved as 300, and applies a maximum cap of 61 to each number, due to the one digit limit.

01_ Base Stat Data:
01_A.2.1s.Z8.1i.D.0.0.uy.jK.jP.iU.ik.20z.2M.0.1C.2X.Z8.uy.0.0

The Base Stats as recorded on the Base Stats tab. Shocking. Having seen the v3 Base Stats tab in previous Dev Log posts, you will have noticed the much longer list of stats it now requires for proper calculation. This is why this section of the URI is actually longer than the previous, despite the much more efficient format.

Each value is separated by a period, which is probably immediately obvious. What isn't so obvious is the order that they are in. The Armory itself also happens to face this issue. The very first value of the set, A in this case, declares the format the data is in. Effectively, this is just telling it what order everything is stored in.

The next value is the character ID, stored in Base 62 (Expect nearly everything to be stored in Base 62 whenever possible...). This one is required for the Armory to reference the correct set of Base Stats, and also happens to conveniently resolve a standing issue where Twin Sword sets load as Lann even if they are from a Vella set. The value after is the character level, used for gear/skill/stat requirements and VIP bonus calculation. The values after are just the stats, and I doubt that the order they are in is all that interesting. Just know that they are indeed all in Base 62, and capped to 3 digits as a failsafe (which is still a maximum value of 238,327...)

02_ Skill Data:

This will be Skill Rank Data, but I have not yet constructed anything final code-wise. Storing these is rather simple comparatively, at least. The final format will likely be this one:
02_##R##R##R

Where ## is the skill's ID, always set as a two digit Base 62 number (since the skill ID may actually exceed 61 in the future), followed by the skill's rank value as a single Hexadecimal value (16 ranks, 16 digits, its perfect).

Since the skill rank data is always three digits per skill, this both allows for it to not care about the order the data is in, AND allows me to omit max skill ranks, as previously mentioned in other logs about space efficiency.

---

I have nothing at all ready to talk about/demonstrate for the remaining four sections, so I'll save that for the next Dev Log covering URI.

Incidentally, this is as far as I can go with coding the Set Link right now. To make it any further constructing these, I need to make the Gear tab functional. To make that properly functional, I must also in turn make the Item List tab, the Right Click Menu, and the Preview Window functional. Of these, two of them don't even exist. Fortunately, the Item List actually exists AND properly generates item lists, albeit with no events attached.

My next thing to do is going to be creating the Preview Window and enabling its functionality. The sheer amount of custom CSS and HTML it has is extreme, so I don't know if I want to keep it exactly as is, despite how good of a job it does at emulating look of the ingame tooltips. I may create a v3-style tooltip using just the basic templates and see how it looks, before I commit to one style or the other.

Due to this bit of tough decision making to do, no ETA on the next Dev Log, except Saturday at latest.

October 7

This Dev Log could actually have been posted on Wednesday, if only skill requirement text display didn't turn out to be such a challenge to work out. On Wednesday, all of the major parts were operational except it.

I decided to drop the HTML/CSS of the remodelled v2 Preview Window into v3, and see how it looks/feels. The restructured code was assembled a lot like the overall v3 code, and didn't need much adjustment to port. ...And upon doing so, scrapped any thoughts of making a completely new Preview Window in v3 style, since despite the slight inefficiency of the CSS for v2, it fits perfectly.

So having adjusted that code, and inserting the associated text into v3, it simply worked straight away. Except for skill rank requirements. The new setup no longer "fakes" skills by using the minimum data possible, and actually has the entirety of the data associated for each skill and skill rank, via using Archives data. The catch is that instead of the emulated "Weapon Mastery" that I had been skating by with, I now have a mastery skill for every weapon type, and the item data itself does not correctly sync to these "true" skill reqs.

So what did I do? I slightly changed the format skill requirement data is stored in, on item data. And then I updated all 1395 item listings to use it. This included updating every skill ID to the actual skill IDs it now uses. As a bonus, I even got to optimize a tiny piece of code in the Preview Window data to only have to loop once instead of twice for fused items. Yay for object merging. It even removed an annoyingly complex function for assembling the skill names for weapon skills, since it was only faking their existence before.

Complaining about that one issue aside, the Preview Window is now fully operational on the Item List, and looks exactly like it currently does on the live site.

From here, the next step is going to be to create a mechanism to mark a gear slot as "selected", so that the Item List and Right Click Menu (which doesn't yet exist) will know what they are modifying. Once that is set up, it will be time to finally work on the Gear tab itself. Both its ability to have equipped items, and its ability to store the data of them for the Total Stats tab and Set Links to make use of.

October 15

This week had two primary challenges: one easy, one quite difficult. Making the Gear window actually function, and stat calculation use equipment, was of course the easier challenge. Devising a format for the set link is the true challenge. I believe that I have found the system I actually intend to use, though. Small changes may still occur, but overall, I am happy with it (until something throws a wrench in it when it comes to its flexibility).

To dive right into the deep end, here is a +15 5* Adjusted Valor Piasraspear infused with +2 Critical, and to compare it, the current v2 link version below it:

3.0545.FF010t3v1
`3`+(`id`+314,`e`+15,`p`+`p01`,`s`+`s29`,`q`+5,`i`+[3,2])

Yes, I saved THAT much space. Let's take a closer look at the v3 piece, and reveal the magic. First, a template mockup:

S.---Q.#EPPSSIII

S - The slot ID it is equipped to
. - The section divider for each slot.
--- - The item ID goes here, in Base 62. (Max supported value 238,327)
Q - The quality of the item
# - The format ID for the rest of this section. This is a value generated via binary (yes/no) for each property to be included, that is then read as a decimal and converted to Base 62.
EX: In this case Enhance, Prefix, Suffix, and Infusion are present. This returns 1111 in binary, which represents 15 in standard decimal format, converted to F in Base 62.
E - The enhancement of the item, in Base 62.
PP - The prefix scroll ID of the item, in Base 62.
SS - Same as above, but for the suffix scroll.
III - The infusion applied. The first I is the stat ID, in Base 62. The second I is the infusion value plus 1921, in Base 62. Why the plus 1921? Negative values. This provides a range of possible values from -1921 up to 1922, without having to deal with the length variance of having or not having a negative sign.

But it goes further! There is an additional parameter set that can be present after quality. Every character after the first four in the second section is treated as an alias for the item's properties. To give an example of what I mean, let's look again at the first item link, and then at an aliased one right below it:

3.0545.FF010t3v1
1.0545vy.F

You will notice that a vy appeared after the quality value (5), and that everything but F (The enhancement value) in the second section vanished. Unrelatedly, the format ID is also gone; this is because in these circumstances, the sole value is always enhancement (more on this in a future Dev Log).

As for the absence of the Prefix/Suffix/Infusion data block, their values were represented by the aliases v and y, allowing them to be omitted. v represents Adjusted Valor, and y represents +2 Critical. Any combination of aliases can be applied to an item, allowing reductions in the required URI size to represent the data.

The functionality of this actually ended up quite complex. For aliases to be meaningful at all, they had to reduce the URI length by at least one, or I'd be replacing two characters with two, accomplishing nothing. However, as a single digit Base 62 value, this also limits me to 62 aliases. That sounds like a lot, but even with just popular prefix, popular suffix, and popular infusions, it was far too few.

The solution? The scope of each individual alias was reduced, to cover either enchants, infusions, or materials. Further to this, aliases were split into five groups, with the item's slot and class dictating which alias group it uses. In effect, this uncaps me to 62 aliases per group, instead of total, allowing much more flexibility. In addition, these more numerous aliases can now be stacked to achieve nearly the same efficiency with fewer actual unique aliases required. This also allowed me to set aside roughly 20+ aliases of each group, for future ES/infusions as they are released.

But enough about Piasraspear, those are easy. What about +15 3* Righteous Valor Regina Twin Spears, infused with +2 Critical, with completely maxed materials? For reference, the v2 link is:

`3`+(`id`+1133,`e`+15,`p`+`p52`,`s`+`s29`,`q`+3,`i`+[3,2],`mats`+[[7080,7080,4,600],[30,31,350],[45,44,24,59,32,350],[21,5,350],[4720,4720,66,35,89,48,350]])

3.0ih33y.F.p8p849G995Ediand5E645EgKgKqdAj5E

...That's not a joke, either. The item ID is 0ih, representing 1133. The first 3 is its quality value of three stars. The second 3 is the first alias, representing Righteous Valor. The y is the second alias, representing +2 Critical. The F is once again +15, and the huge block in section 3 is the composite stats. So what is going on here, you might wonder.

Firstly, it's important to note that every composite weapon has the same five materials, with the same maximums and minimums. This stays true for every armor slot, and every shield, as well. Knowing this, instead of storing the direct value for each part, I can store the difference between the current value and the minimum. In several cases, this reduces the required digits by one, making the extra step absolutely worth it.

Since these materials are not subject to change down the line, I can safely apply the minimum possible digits to each stat on each material, and that is exactly what I did. Since the exact length of everything is always exactly the same, despite how illegible it looks, it is extremely tidy in the eyes of JavaScript. I can simply pull off substrings for each stat sequentially. Oh, and just to bring up aliases again:

3.0ih33y.F.p8p849G995Ediand5E645EgKgKqdAj5E
3.0ih33yB.F

That is absolutely going to be a thing with the presets. Due to the redundant INT/M.ATT values on STR gear and vice/versa, it is unlikely that any exactly set gear will match these preset aliases, but I cannot bring myself to remove M.ATT off the materials. It would feel wrong, and it would set off my very slight OCD to allow a technical calculation error to exist, even if only on the Preview Window.

Sockets will work on a similar mechanism, except no presets will be allocated to bracelets due to their sheer variance, and I have not yet devised a method for storing them other than brute forcing via 7 digits of Base 62 per socket.

---

Despite having planned out all of this detail for how item URI generation will go, this code does not actually function yet. The format ID was a huge roadblock for many reasons, and I am still not entirely sure that the solution I have in mind will actually work for ALL items as it is required to.

As such, my next goal is to successfully generate a URI piece for each type of gear (Lv.90 weapon, Lv.90 armor, socketed bracelet, regular gear piece, a necklace, an S-rank avatar piece, and a nonexistent test item that has both sockets and composite materials, simply to test its ability to adapt to new conditions.

Starting Monday, I will be out of state for a week or so. I have a laptop and so will still be reachable/etc, but this will mean that I will not be as active with development. As such, there will be no Dev Log for next week. Progress will be made, but I don't want to have people expecting a Dev Log that would very likely say "Still working on it, nothing to talk about yet".

October 29

I'm going to make a one time exception (hopefully...) for this Dev Log, and say check back tomorrow.

So far, every day since I've gotten back from my trip on Sunday, I've had some distracting bit of illness impacting my ability to get things done. This slowed development, especially combined with the Armory patch I had to push out this week to adjust Effective ATT calculation on the live Armory.

Excuses aside, later today, probably near 24 hours from now, I'll have a proper Dev Log talking about the set link generation actually working. As is, there are still formatting incompatibilities on composite material data, and I will be implementing another step of "aliasing" to reduce more variations of material stats, due to the reality that very few weapons will have perfectly maxed rolls since ATT and M.ATT are redundant, just as STR and INT are redundant.

It still won't be on the level of showing a full generated link, but I am getting there. Once it can generate individual items, I can start making the Gear tab work to actually make a full set. If lucky, maybe this very same week?

October 30

As promised yesterday, today's Dev Log is about the successful implementation of URI generation for items. Only one at a time so far, due to not having a functional Gear tab, but this is a very important step nonetheless. No URI linking means no storing or sharing sets, which is no fun.

I've already covered everything of interest about normal equipment, so let's jump straight back to the +15 3* Righteous Valor Regina Twin Spear w/+2 Critical example. A fully decked Lv.90 weapon, with lots of data that needs to be handled. For reference, this is how v2 handles it:

`3`+(`id`+1133,`e`+15,`p`+`p52`,`s`+`s29`,`q`+3,`i`+[3,2],`mats`+[[7080,7080,4,600],[30,31,350],[45,44,24,59,32,350],[21,5,350],[4720,4720,66,35,89,48,350]])

I've already shown examples on the 15th of roughly what this particular example would look like, but here it is in its actually functional format:

3.0ih33BE.f

You'll notice that the only actual differences are the alias IDs themselves. 3 denotes the Quality, B denotes Righteous Valor, and E is the Super Max Rolls preset's alias. Enhancement is still the lone occupant of the second block of data.

But if you are particularly clever, you might be aware that this isn't exactly a fair representation. How many people are legitimately going to max both ATT and MATT, as well as STR and INT on their weapon on every material? It's a waste of resources, after all. So let's change the example a little bit. Let's make the M.ATT on the Feather by 6881, and the M.ATT on the Perfect be 4588. That's sufficient to prevent my Super Max Rolls alias from applying, so let's look at it now:

3.0ih33B.f.p8lV49G995Ediand5E645EgKeCqdAj5E

Back to a long form version, with no aliasing. Right? Nope, actually not! I can pull more aliases out of my ass, and do this:

3.0ih33B.f.7uC_p8lV49GgKeCqdAj5E

It's not a major reduction, certainly, but I did reduce it, and every bit helps. So how does this one work?

The method of generation for each stat is the same as mentioned in the October 15th Dev Log, that hasn't changed. However, before actually being combined into the full string, the URI data is stored in an array, separated by each material. I can iterate through a group of material aliases and check for any matches within this array. If a match is found, it tacks it onto the list of material aliases, and removes that material from the array. In this case, the Keen, Stable, and Lightweight fit the Super Max Rolls aliases, and were stored as 7, u, and C respectively. Since the order and length of the materials is the same, reading the URI, it simply needs to know which mats were aliased out to skip them when parsing the data.

Since there will definitely not be future changes expanding the stats of the Lv.90 materials again, I can make full use of the 62 aliases for each of weapons and armor, to cover a large amount of possibilities. Primary mats, Perfects, and Stables will still pretty much only match if their normal stats are fully maxed, or fully minimum Top Grade stats, but every variation of Top Grade Keen and Lightweight shards is represented, meaning they will be aliased 100% of the time. Nearly every variation of Remove ATT Limit is covered as well, due to the unlikelihood of a mat being completely maxed on it.

This Dev Log only covers a weapon example since at the moment the armor aliases do not function. However, this won't take long to correct, and is not worthy in itself of a Dev Log. The next Dev Log will be covering the Gear tab, unless something goes wrong or sidetracks me on a bigger goal.