spotmarket Graph SamplePosted: 2016-06-07 Filed under: nullsec, pvp, python, spotmarket | Tags: amarr, branch, deklein, dodixie, fade, guristas, jita, pure blind, rens, tenal, the forge, tribute, venal Leave a comment
What’s data without some graphs? Here is a small snip of the graphing that is currently in the spotmarket project.
2016-01-13 to 2016-06-07
2016-01-13 to 2016-06-07
Jumps in Trade Hubs
Jumps and NPC Activity in The Northern Regions
Gurista Ratting Rates
NPC Universe Overview
First pass at a map using D3.js inspired by HelicityBoson at http://www.machine9.net/?p=1111.
spotmarket – 0.4Posted: 2016-03-18 Filed under: python, spotmarket | Tags: amarr, dodixie, fountain, jita, PLEX, rens Leave a comment
Whew, this was a big step in the right direction. I added a lot of sanity (keep everything in UTC), usability, and cleanup to the existing design. We’ve got moongoo, the beginnings of indexes, and CREST killmail data now flowing in to the growing dataset. The backend was improved by migrating to PostgreSQL 9.5.1 and I also cleaned up the install directions to try to make it easier.
[bug] Change timestamp for zkillboard and markethistory consumer to use UTC for mental sanity
[enhancement] Add 404 error page
[enhancement] Change to postgres 9.5.1 to support jsonb
[enhancement] Parameterize graph functions
[enhancement] Change date format in graphs to ISO for mental sanity
[enhancement] Standardize table formatting
[enhancement] Supervisor to make Flask web service persistent
[enhancement] Add jsonb index on data.killmails for typeID and solarSystemID
[enhancement] Add paging to zKillboard consumer
[enhancement] Add check to resume from last recorded page for zKillboard consumer
[enhancement] Add basic exception handling to zKillboard consumer
[new] Creation of ship index report
CREST Verified Moon Minerals
It is no secret that I have an affinity for finding accurate moon data. In a previous post, I’ve gone so far as to chart the regional density of each moon mineral, showing that there were built in skews as to where each type was placed for each class (R8, R16, etc).
In the Rubicon expansion, CCP introduced siphon units, which can be anchored near a POS to slowly leech items from within the shields. When an extractor is destroyed the killmail will show what moon mineral was in it. This, combined with the x,y,z location data that started to be exposed after the Parallax expansion, can be used to locate the nearest celestial object to give us a verified report of what moon mineral is contained in the object.
I gathered some examples for anyalisis, and over a long flight, I started to put some code together to use killmail x,y,z coordinates to parse CREST killmails. After implementation and testing, it has proven to be accurate so I’ve included the feature in this release. There is a manual step to get the moon mineral data into a table where it is used in the web frontend; I wanted to keep them separate to isolate the two datasets.
PostgreSQL Upgrade to 9.5.1
Rather than writing a parser for CREST killmails, I decided to store the JSON itself for simplicity. Developing on 9.3 with the json datatype proved to be a path that I did not want to go down. 9.3 is not the prefered version and lacked support for a fancy datatype inherit to 9.4+ and above. The following blog post also convinced me to migrate.
The proof is in the numbers so here’s a side-by-side comparison of the same query on the same dataset with only the datatype being different. The two virtual machines also had access to the same amount of CPU and RAM to control the results.
This query was proof enough that I made the right choice. Later that night while checking my logging table, I also noticed an improvement in insert speed.
9.3 (json) [kills] insert 2982 @ 93.81 rec/sec 9.5.1 (jsonb) [kills] insert 2850 @ 124.44 rec/sec
Not bad for about 45 minutes of testing and correcting install instructions.
What’s up for the next release? Check out the TODO.md for a full list of items slated for each release. The main focus is going to be putting more control in the web front end, letting you enable/disable import items, add item to the market/zKillboard watch list, etc.
Rorqual MovePosted: 2014-12-09 Filed under: ships | Tags: amarr, gallente, rorqual 3 Comments
I spent a few hours moving a Rorqual to my new home system and almost got caught making a fatal mistake because of my lack of knowledge regarding standings if you are in Factional Warfare. It turns out that standings can affect where you dock when you are in FW.
Since this was my first capital operation since the Phoebe expansion, I wasn’t prepared for the amount of jumps Dotlan pulled it. I was thinking that it was going to be a simple 2-3 jump operation, but actually came out to 7.
Everything was going according to plan until I hit the Ofstold system, which is an Amarr system.
[ 2014.12.06 06:40:34 ] (info) You have been denied access for the following reason: The Amarr Empire denies access to Factional Warfare enemies. Either assist the Minmatar Republic to capture this system or retire from the war.
I paused for a second, let out an audible curse, and checked local — empty. The quickly formulated backup plan that came to mind was to warp to the sun, create a safe inline, and warp to the newly created safe. If someone came into local, I was going to log off and wait them out until I got my cyno character in position in the next system.
I was able to get my character in position in the next system, wait out the 12 minute fatigue timer, and safely land on the next station.
Even if you have been playing Eve for 5 years, there are still areas of the game that surprise me. I was quite lucky that the system that I could not dock in was currently empty.
Historical Profits by Solar SystemPosted: 2014-04-18 Filed under: industry, nullsec | Tags: amarr, G-TT5V, G8AD-C, highsec, jita, lowsec, nullsec, Otsela, VFK-IV 1 Comment
CCP has released two out of the six Devblog posts aimed at industrialists detailing changes for the upcoming Summer expansion. We’re seeing sweeping changes to the way logistics are done for capital ships, station research, POS anchoring limitations, BPO security concerns, and how inventors are going to be given a boost with BPC copy rates. Lockefox has a great summary of the concerns in his Everything Is Changing post.
Given the ideology coming out of the Dev Blogs to empower Nullsec industrialists while kicking Lowsec in the knees, I wanted to see how much of my industrial gameplay occurs in Lowsec — Is CCP killing my game?
Summary of profit grouped by solar system with profit numbers obfuscated.
— Get Profit per Station By Solar System including Region
SELECT SUM(profit), mapDenormalize.itemName, mapSolarSystems.solarSystemName, mapRegions.regionName, AVG(mapDenormalize.security)
ON (wallet.stationID = mapDenormalize.itemID)
ON (mapDenormalize.regionID = mapRegions.regionID)
ON (mapDenormalize.solarSystemID = mapSolarSystems.solarSystemID)
GROUP BY wallet.stationID
ORDER BY sum(profit) DESC
It turns out that Lowsec only accounts for 7% of our profits to date so I can’t complain about the nerf that is going to hit Lowsec capital builders given the upcoming compression changes.
The changes to compression are a welcomed change, even if it means retiring or heavily modifying the logistical chain for Lowsec capital production. I have a feeling that there are going to be more major changes in the next four upcoming Devblog posts. Stay tuned.