Author Topic: The new GT  (Read 5304 times)

Offline bontchev

  • ****
  • Posts: 408
    • View Profile
The new GT
« on: 15 August 2012, 05:34:05 »
WTF happened with the GT pages?! I no longer see a thing! Just the background with some tiny strip at the top!

I was able to see the beta version while it was still in development just fine!

Offline bontchev

  • ****
  • Posts: 408
    • View Profile
Re: The new GT
« Reply #1 on: 15 August 2012, 07:57:31 »
Folks, you have totally, utterly screwed probably the most useful tool for the game! :'( :'( :'(

The problem mentioned above still doesn't occur with the beta site. So, it's probably related to the advertisements. But it doesn't disappear when I turn my ad blocker on for this site, so it must be deeper than that. It also doesn't appear in other browsers (IE, Chrome, Opera). Are you using some advanced CSS (like 3.0) in the ads-related sections, by any chance?

OK, so I used other browsers to access the site. It still doesn't work!!!

To begin with, all my saved searches are GONE! "Form loading error" says the attempt to open their list.

OK, so I created new ones. For instance, I like to search the whole universe for debris fields larger than 20.000. So, I created such a search and it seemed to work fine, so I saved it. Then I loaded it, executed it and suddenly nothing was found!!! Apparently, the problem is related to the comma-separating sign. If I enter "20000" manually again, the search works - but the number is displayed as "20,000" (yes, with a comma - not with a dot) when I load the search.

Next, I created a search for the spy reports. I want to search galaxies 1-5 for inactives of any kind, not in vacation mode, with at least 250.000 total resources, less than 800 in defenses and spied within the last 24 hours. If I specify all that, I get an empty list!!! Yet there must be many dozens of such reports!

I finally nailed down one of the problems to the age of the reports. If I don't specify "24", I get more than 0 hits. Except that I get old reports too, of course - which isn't what I need. What is the problem here? And, while it is nice to be able to specify "only the reports from the last N hours", what happened with the control that let you select only the reports between to given times? Adding new functionality is nice, removing the existing one isn't.

Note that I said "hits" above - not "reports". That is because I didn't get any reports!!! Instead, I got "sorry, an error has occurred"!!! This happens in IE and Opera (latest versions both). Since you no longer support FF 3.* at all, my only option was to try Chrome. I did get the reports there, and the look is much nicer too - not so dark and unreadable like in the other three browsers - but, come on, folks, not everybody uses Chrome! And the report age problem still remains, making the output list utterly unusable to me. :'(

Is the old version available for download from somewhere? I might be forced to install it on my own host, because the current one just doesn't cut it. :'(

Offline bontchev

  • ****
  • Posts: 408
    • View Profile
Re: The new GT
« Reply #2 on: 15 August 2012, 13:17:42 »
Even when using Chrome, this thing is unusable!

In the old version, the output of a search in reports had nice tooltips popping up and telling how how many ships you'd need, in order to plunder this planet (both kinds of cargoes and battleships), and how many recyclers to recycle the fleet. Where are they now? Nothing pops up there, at least not in Chrome...

Not to mention that the 200-report limit truly sucks. I scan 250-350 inactives every day. I want to know which ones of them are totally defenseless, so that I can attack them with cargoes only overnight. There are more than 200 of those. I wouldn't mind if only the 200 richest ones were displayed - but the tool doesn't do that, does it? It just displays the first 200 and then you can sort the displayed results by amount of resources. So, I'll have to have multiple scans saved, one per galaxy, in order to fit in this silly limit... It won't reduce your traffic (and might even slightly increase it) - it will just inconvenience me... :-(

And speaking of inconveniences, if I update the data using the game API, the Database Status page says that all the entries are new, which is as it should be - but why the graphical representation (Galaxy Overview) says that all the entries are old? I think this is just a bug - you have forgotten to update the status there.

And speaking of updating via the game API - you have updating of the whole universe or of the statistics only. How about a third option - the player statuses only? It changes every day and greatly helps to find new inactives.

There are many, many other annoyances (some of them I've written about in the past, during the testing phase) - but that's of secondary importance; first we have the make the thing actually work!

Offline bontchev

  • ****
  • Posts: 408
    • View Profile
Re: The new GT
« Reply #3 on: 15 August 2012, 14:38:31 »
Folks, I'm getting the distinct impression that your date comparisons don't work because you don't take the locale formats into account...

I tried Search in Database for players whose information is newer than 15-Aug-2012, 00:00 (i.e., from this morning). The date/time pickers selected that as "8/15/2012" and "12:00 AM" respectively. (I hate the American date/time format, BTW.) The search yielded 0 results, which couldn't be right, since I viewed a bunch of systems this morning, while spying the inactives.

Then I ran another search for planets of inactives with no spy reports. This yielded 16 hits, all of which have today's date - except that it is listed as "12:30 15.08.12", which is the German format used by the game everywhere (even on the US servers).

Offline bontchev

  • ****
  • Posts: 408
    • View Profile
Re: The new GT
« Reply #4 on: 15 August 2012, 15:17:21 »
And why, despite me having updated "All data" from the game API, some alliances are listed in the Statistics as having 0 Economy, Fleet, Research or Military (all kinds) points? Arrgh, give us back the old version! >:(

Offline eX0du5

  • Administrator
  • ****
  • Posts: 560
    • View Profile
Re: The new GT
« Reply #5 on: 15 August 2012, 17:17:30 »
WTF happened with the GT pages?! I no longer see a thing! Just the background with some tiny strip at the top!

I was able to see the beta version while it was still in development just fine!

This occurred because of outdated caches in your browser. I pressed Ctrl+F5 to refresh the page and then it worked fine. I don't know why this occurred only at my second PC (Firefox) but not yesterday after retesting for the same tool just after the update.

Regarding the other points you raised, I will check them later when I am back @home in detail to see what is wrong there.
Did you encounter the same issues in the beta as well? Because I fixed all known issues which were reported there.

Offline Omar Hawk

  • Administrator
  • ****
  • Posts: 900
    • View Profile
Re: The new GT
« Reply #6 on: 15 August 2012, 18:19:54 »
And speaking of updating via the game API - you have updating of the whole universe or of the statistics only. How about a third option - the player statuses only? It changes every day and greatly helps to find new inactives.

The players.xml is updated once a week.

Offline eX0du5

  • Administrator
  • ****
  • Posts: 560
    • View Profile
Re: The new GT
« Reply #7 on: 15 August 2012, 21:34:46 »
I will go through the findings and update the status here.

I found the problem for report save - which worked fine last time i tested save & load. Since then I tested either save or load :-(
It was simply this wrong html code:
Code: [Select]
<input id=fleet_resis"This caused problems and made the json response inconsistent :-(
This is not fixed and you should be able to store and load the data again. Inconsistently saved data must be deleted.


Next regarding the form saves - they are gone because I decided to let them go. This was a technical and effort related decission which I took somewhen back. I do not want to go into details but I had the decission to write a (not 100% working) upgrade script or skip this to have an incompatible approach to grab and store the content.

I cannot reproduce problems with loading 20,000 or 20.000 depending on my locale. I switched them and checked both. The data sent to the backend is 20000 in all cases no matter what is displayed in the form field.

The other thing with the search within last x hours has also been fixed.

Yes, you can still download 4.9.2 in our wiki page. It has not been removed yet.

The reports still show the popup but with the small icon on the very left page. It will show the report with all the new features you requested. (e.g. too much resources for cargo)
The report results are now considered to display already the information you need for your attachs and you do not want to attack more than 200 planets at once but you will just attach around 20. Maybe you need 200 hits to find the 20 you want,but you can always narrow this down with the search options until you finally have the results you want to attack. It is no longer a "show me all you got" but more intended to guide you what to farm.
Speaking of the reports, you can press the small icon on the right in your search results to configure the columns you want to see there. Therefore you do not need a small mouse over of the reports to quickly check whats in the report. You have all data in your columns if you need them.

« Last Edit: 15 August 2012, 22:46:49 by eX0du5 »

Offline bontchev

  • ****
  • Posts: 408
    • View Profile
Re: The new GT
« Reply #8 on: 15 August 2012, 22:17:12 »
This occurred because of outdated caches in your browser. I pressed Ctrl+F5 to refresh the page and then it worked fine. I don't know why this occurred only at my second PC (Firefox) but not yesterday after retesting for the same tool just after the update.

Ahhh, thanks a bunch! Yes, it works now. Didn't even have to press Ctrl-F5 - I guess the cache expired all by itself. This used to be a problem with the game, too, but they figured out a way to instruct the browser to flush the cache when a new version is released. I don't know how this is done; if you do, please consider doing the same for the tool.

Quote
Regarding the other points you raised, I will check them later when I am back @home in detail to see what is wrong there.

If you want, let's agree on a time for a real-time chat via something - IRC/messenger/whatever.

Quote
Did you encounter the same issues in the beta as well? Because I fixed all known issues which were reported there.

I couldn't encounter the same issues there, since I couldn't actually use the beta - it was for a universe in which I don't have an account. So, I played with the user interface and that was it. But now that I have to use the tool in the universe where I actually play and rely on the data provided by it, all sorts of issues started to appear...

Perhaps the next time you could temporarily remove the universe restriction in the beta or set up a beta account for the universe in which I actually play?

Offline bontchev

  • ****
  • Posts: 408
    • View Profile
Re: The new GT
« Reply #9 on: 15 August 2012, 22:21:56 »
And speaking of updating via the game API - you have updating of the whole universe or of the statistics only. How about a third option - the player statuses only? It changes every day and greatly helps to find new inactives.

The players.xml is updated once a week.

I am not sure what you mean. The game updates that file no more often than once in 24 hours. That is, it doesn't update it for 24 hours and after that updates it only when somebody requests it. Trust me, it's not once a week. I've written a PHP tool that uses this file to locate new inactives and outlaws every day.

(Similarly, universe.xml is updated no more often than once a week and highscore.xml is updated no more often than once an hour.)

If you mean that GT updates (reads) this file once a week automatically - that's not what I meant, either. First of all, it's not often enough - see above. Second, in Administration, there is a button for updating the universe, a button for updating the statistics - so, I'm asking for a button for updating the player list.

Offline bontchev

  • ****
  • Posts: 408
    • View Profile
Re: The new GT
« Reply #10 on: 15 August 2012, 22:44:08 »
This is not fixed and you should be able to store and load the data again. Inconsistently saved data must be deleted.

I am not sure I understand you. Are you talking about the saved searches? Yes, they seem to work correctly now.

Quote
Next regarding the form saves - they are gone because I decided to let them go. This was a technical and effort related decission which I took somewhen back. I do not want to go into details but I had the decission to write a (not 100% working) upgrade script or skip this to have an incompatible approach to grab and store the content.

OK, but perhaps a little warning would have been nice. :)

Quote
I cannot reproduce problems with loading 20,000 or 20.000 depending on my locale. I switched them and checked both. The data sent to the backend is 20000 in all cases no matter what is displayed in the form field.

Perhaps this was related to the problem of saving searches, because now it works correctly here too.

Quote
The other thing with the search within last x hours has also been fixed.

Ahhh, yes! Thanks a lot! This was one really major issue for me - without it the results were simply unusable, since I had no way of throwing out old reports. I usually scan 250-350 inactives (for which I've determined that they have at least a metal mine level 20, non-zero energy and less than 800k defenses) every morning and filter those of them that currently have more than 250k total resources. With the "within the last x hours" thingy not working, I couldn't tell the search to show only the results from "today".

Quote
Yes, you can still download 4.9.2 in our wiki page. It has not been removed yet.

Yes, found it. Don't remove it for a while, in case other people have problems with the new version, too. There are still a lot of things that need improvement in the new version - but, as I said, that's of secondary importance. Let's fix the major bugs first.

Things that still remain:

1) Despite loading "All data" from the game API from Administration, the Statistics page shows alliances with 0 Economy, Fleet, Research and Military points.

2) When a Search in Reports is performed, there is a very nice control that lets you select which columns to show. That control has "Check All/Uncheck All" links. They don't work consistently. Like, "Check All" doesn't check everything in the current section (but might check things in another section) and "Uncheck All" doesn't uncheck everything in the current section.

3) The Search in Reports result don't have tooltips showing the number of ships and recyclers needed to plunder the target. This is a major problem for me.

4) Search in Database cannot search by specified date. I mean, if I specify a date, nothing is found, although there ought to be some entries. I still think that this is a locale problem. Perhaps we could investigate it interactively?

5) Database Status -> Galaxy Overview doesn't show that all entries are new after an update via the game API. Just a minor inconvenience.

Offline bontchev

  • ****
  • Posts: 408
    • View Profile
Re: The new GT
« Reply #11 on: 15 August 2012, 22:53:41 »
Oh, drat, another major issue. :( When passing the data to WebSim, you don't pass it the coordinates of the target, so it cannot compute the needed fuel. :(

Offline eX0du5

  • Administrator
  • ****
  • Posts: 560
    • View Profile
Re: The new GT
« Reply #12 on: 16 August 2012, 00:09:05 »
Just as update. All points are fixed except the tooltip at report search that shows the required ships for debris and resources.

Offline bontchev

  • ****
  • Posts: 408
    • View Profile
Re: The new GT
« Reply #13 on: 16 August 2012, 19:11:45 »
Thanks a bunch! :D I've been playing with it the whole day and, yes, indeed, after the fixes you did the last night, it has become very usable. There are still a whole bunch of bugs, annoyances and lack of features left - but they are minor; at least there is no longer anything left that prevents me from using the tool.

For the record, here is what I've found so far:

Bugs:

1) In Galaxy view, there is no indication that the IPMs of the current user can reach the currently viewed system. This is an old bug - it already existed in the previous version, but I thought that it was supposed to be fixed already.

2) There is something strange with the saved phalanx times. Aren't they supposed to be in server time? Because they were in my local time. I use AntiGame, which is configured to convert them to local time in the game and I have also set a timezone +2 for my GT account - perhaps this matters? I don't know how other GT users see them (i.e., as local times or not, and whether they see the same times as I do).

3) The "Manage columns" control of the Search in Reports output sports a scroll bar when its sections are expanded to become longer than the cntrol's height. This is as it should be, of course. The problem is that if you expand all sections (or even just most of them), you can't use the scroll bar to scroll down far enough, in  order to see the "Apply" button. This is probably related to the fact that you have added new entries in these sections in the last moment.

Annoyances:

1) The buttons for saving or loading a search (both in Search in Database and in Search in Reports) currently reside in the topmost section. This is wrong, because as soon as you perform a search, all sections collapse, including the first one, thus hiding these buttons. So, you have to open it manually (one additional mouse click), if you want to save the current search or to load another. Please consider moving these buttons above all the sections where they are not obscured when all the sections are collapsed.

2) When you pop up an espionage report (by clicking on the looking glass icon), the only way to close it is by hitting the [x] icon in its upper right corner. It is rather small and difficult to hit with the mouse. I'm not sure what a good solution would be, but please give it some thought. For instance, make the button larger (e.g., a long button with "Close" written on it)? Or closing the report if the user clicks outside it, as the game does?

3) This one is a bit difficult to explain... When Search in Reports produces a lot of results, they don't all fit on my screen. So, I have to scroll them. I like to have in the middle of the screen the line I am currently working on (e.g., looking at its espionage report, in order to call the simulator, copying its coordinates, in order to paste them in the game, etc.). So, I scroll the results until that line gets in the middle of the screen. Now, if I switch to another browser tab (e.g., to the game, where I launch a fleet, etc.) and then back the GT tab, the selected line doesn't remain in the middle of the screen - it scrolls back to where it was before I put it in the middle. This is rather annoying. Can it be fixed?

4) As I've told you during the beta testing phase, the current color scheme is too dark and difficult to see on my screen. (Interestingly, it looks much better in Chrome - but this isn't the browser I'm normally using.) Yes, I realize that the game uses a lot of dark backgrounds too - but it puts bright texts and pictures on them, thus creating a contrast and making the important information easy to see. Much easier than the information in GT. Unfortunately, I am totally devoted of artistic sense, so I can't tell you exactly how to fix this problem. :( I can see that the color scheme is not good - but I have no idea what a good color scheme would look like. Is there a professional designer reading this forum? We really could use some help here... At the very least, the coordinates should be white, as they are in the game - not bluish-gray, as they are now. But this is only one obvious problem, fixing just it is by far not sufficient to resolve the general issue...

Improvement suggestions:

1) Color the names of the players in Galaxy view and in the statistics to reflect their status. At the very least color the inactives and those in vacation mode. Use the same colors that the game uses. Ideally, also color the noob, strong, honorable and neutral targets the way that the game colors them in the Galaxy. This might be a bit more difficult, because you'll have to compute the status of the target relatively to the current user - while the inactives and those in vacation mode should be colored always, for all users.

2) Include the player name in the pop-up espionage report the way the game does. (Resources at <planet_name> [<coordinates>] (<player>) on <date_and_time>)

3) When viewing the information about a player, you show the phalanx ranges covered by this player's phalanxes. Please also add the IPM ranges covered by this player's IPMs.

4) When viewing an espionage report, indicate not the sum of all resources, but the sum of plunderable resources and also the number of cargoes (small and large) needed to plunder them, as well as the number of recyclers, necessary to recycle the fleet, if any. Note that this depends on the honor status of the target (neutral, honorable or bandit); see below.

5) Add support for the OSimulate simulator. Ideally, let the user specify in the options which particular simulator (of the supported ones) they prefer (most people usually use one and the same) and then have just one "Simulate" link in the espionage report, instead of as many as there are simulators supported.

6) Have ">="/"<=" selectors in the Resources section of Search in Reports. This will let me perform searches like "which inactives that I have spied have less than 70k resources?", so that I can remove them from my list of worthwhile targets.

7) Make it possible to sort the Search in Database output by column header. I'm particularly interested in sorting by total number of debris.

8) Introduce a column selection control to the Search in Database results - like the one you have for the Search in Reports results, only containing the relevant column names.

9) Show the player status (bandit, emperor, etc.) in the Galaxy, in the search results and in the pop-up espionage reports. Yes, I do realize that this isn't easy. The game API does not provide this information (why the heck not?! it does provide the "outlaw" status!), so you'll have to get it from the plug-in - and I don't think that the plug-in currently provides it. Still, please have in mind this request.

10) Allow searching by player status. For instance, it can be useful to find all Bandits in a given range. Again, you'd need cooperation from the plug-in.

11) Account the plunder properly for honorable and bandit targets (75% and 100% respectively). Of course, for that you have to know the status of the target, so it is probably related to the problems mentioned in the previous two points.

12) Please improve the charts in Historical Overview. Now they suck big time. They are even worse than they were in the old version - and they were pretty awful there already. ;D Please consider using a different charting package. (For instance, the one used by the war-riders.de site. Also, I think Google has a pretty decent one.) At the very least use one that knows how to scale properly. Now the scale seems to be constant (from 0 to some large value), so just about any line looks straight with barely a squiggle visible on it. The scale should be from slightly below the lowest Y-coordinate of the line to slightly above the highest Y-coordinate of it. The blue color of the lines is pretty awful and difficult to see, too. About the only improvement I can see is that now there are tooltips, showing the actual values at each point.

Offline bontchev

  • ****
  • Posts: 408
    • View Profile
Re: The new GT
« Reply #14 on: 17 August 2012, 05:35:24 »
Another annoyance/bug: there is inconsistency between the specified Defense score (Search in Reports -> Miscellaneous) and the displayed defense score in the results. This was there already in the old version, I think. Basically, you specify in multiples of a thousand (e.g., "800") while the results display the full number (e.g., "800.000"). I don't mind using multiples to save typing (since all prices of the defenses are a multiple of 1000), but use the same number in both places, please. I haven't tried, but I suspect that the same problem exists with the specified Fleet score.

And the suggestion for improvement already mentioned somewhere above: in Administration, implement a button for loading only of the player statuses (players.xml). That file changes once a day and you might not want to download everything (since the universe data changes only once a week) and the statistics doesn't contain the player statuses (which are necessary to locate new inactives, for instance).