spotmarket Graph Sample
Posted: 2016-06-07 Filed under: nullsec, pvp, python, spotmarket | Tags: amarr, branch, deklein, dodixie, fade, guristas, jita, pure blind, rens, tenal, the forge, tribute, venal Leave a commentWhat’s data without some graphs? Here is a small snip of the graphing that is currently in the spotmarket project.
Data
Jumps
2016-01-13 to 2016-06-07
17,802,640 Rows
Kill
2016-01-13 to 2016-06-07
9,819,421 Rows
Graphs
Jumps in Trade Hubs
Jumps and NPC Activity in The Northern Regions
Gurista Ratting Rates
NPC Universe Overview
Outer Ring
First pass at a map using D3.js inspired by HelicityBoson at http://www.machine9.net/?p=1111.
spotmarket – 0.4
Posted: 2016-03-18 Filed under: python, spotmarket | Tags: amarr, dodixie, fountain, jita, PLEX, rens Leave a commentOverview
Whew, this was a big step in the right direction. I added a lot of sanity (keep everything in UTC), usability, and cleanup to the existing design. We’ve got moongoo, the beginnings of indexes, and CREST killmail data now flowing in to the growing dataset. The backend was improved by migrating to PostgreSQL 9.5.1 and I also cleaned up the install directions to try to make it easier.
[bug] Change timestamp for zkillboard and markethistory consumer to use UTC for mental sanity
[enhancement] Add 404 error page
[enhancement] Change to postgres 9.5.1 to support jsonb
[enhancement] Parameterize graph functions
[enhancement] Change date format in graphs to ISO for mental sanity
[enhancement] Standardize table formatting
[enhancement] Supervisor to make Flask web service persistent
[enhancement] Add jsonb index on data.killmails for typeID and solarSystemID
[enhancement] Add paging to zKillboard consumer
[enhancement] Add check to resume from last recorded page for zKillboard consumer
[enhancement] Add basic exception handling to zKillboard consumer
[new] Creation of ship index report
CREST Verified Moon Minerals
It is no secret that I have an affinity for finding accurate moon data. In a previous post, I’ve gone so far as to chart the regional density of each moon mineral, showing that there were built in skews as to where each type was placed for each class (R8, R16, etc).
In the Rubicon expansion, CCP introduced siphon units, which can be anchored near a POS to slowly leech items from within the shields. When an extractor is destroyed the killmail will show what moon mineral was in it. This, combined with the x,y,z location data that started to be exposed after the Parallax expansion, can be used to locate the nearest celestial object to give us a verified report of what moon mineral is contained in the object.
I gathered some examples for anyalisis, and over a long flight, I started to put some code together to use killmail x,y,z coordinates to parse CREST killmails. After implementation and testing, it has proven to be accurate so I’ve included the feature in this release. There is a manual step to get the moon mineral data into a table where it is used in the web frontend; I wanted to keep them separate to isolate the two datasets.
PostgreSQL Upgrade to 9.5.1
Rather than writing a parser for CREST killmails, I decided to store the JSON itself for simplicity. Developing on 9.3 with the json datatype proved to be a path that I did not want to go down. 9.3 is not the prefered version and lacked support for a fancy datatype inherit to 9.4+ and above. The following blog post also convinced me to migrate.
The proof is in the numbers so here’s a side-by-side comparison of the same query on the same dataset with only the datatype being different. The two virtual machines also had access to the same amount of CPU and RAM to control the results.
This query was proof enough that I made the right choice. Later that night while checking my logging table, I also noticed an improvement in insert speed.
9.3 (json) [kills] insert 2982 @ 93.81 rec/sec 9.5.1 (jsonb) [kills] insert 2850 @ 124.44 rec/sec
Not bad for about 45 minutes of testing and correcting install instructions.
0.5 Release
What’s up for the next release? Check out the TODO.md for a full list of items slated for each release. The main focus is going to be putting more control in the web front end, letting you enable/disable import items, add item to the market/zKillboard watch list, etc.
Rorqual Move
Posted: 2014-12-09 Filed under: ships | Tags: amarr, gallente, rorqual 3 CommentsI spent a few hours moving a Rorqual to my new home system and almost got caught making a fatal mistake because of my lack of knowledge regarding standings if you are in Factional Warfare. It turns out that standings can affect where you dock when you are in FW.
Since this was my first capital operation since the Phoebe expansion, I wasn’t prepared for the amount of jumps Dotlan pulled it. I was thinking that it was going to be a simple 2-3 jump operation, but actually came out to 7.
Everything was going according to plan until I hit the Ofstold system, which is an Amarr system.
[ 2014.12.06 06:40:34 ] (info) You have been denied access for the following reason: The Amarr Empire denies access to Factional Warfare enemies. Either assist the Minmatar Republic to capture this system or retire from the war.
I paused for a second, let out an audible curse, and checked local — empty. The quickly formulated backup plan that came to mind was to warp to the sun, create a safe inline, and warp to the newly created safe. If someone came into local, I was going to log off and wait them out until I got my cyno character in position in the next system.
I was able to get my character in position in the next system, wait out the 12 minute fatigue timer, and safely land on the next station.
Even if you have been playing Eve for 5 years, there are still areas of the game that surprise me. I was quite lucky that the system that I could not dock in was currently empty.
Historical Profits by Solar System
Posted: 2014-04-18 Filed under: industry, nullsec | Tags: amarr, G-TT5V, G8AD-C, highsec, jita, lowsec, nullsec, Otsela, VFK-IV 1 CommentOverview
CCP has released two out of the six Devblog posts aimed at industrialists detailing changes for the upcoming Summer expansion. We’re seeing sweeping changes to the way logistics are done for capital ships, station research, POS anchoring limitations, BPO security concerns, and how inventors are going to be given a boost with BPC copy rates. Lockefox has a great summary of the concerns in his Everything Is Changing post.
Given the ideology coming out of the Dev Blogs to empower Nullsec industrialists while kicking Lowsec in the knees, I wanted to see how much of my industrial gameplay occurs in Lowsec — Is CCP killing my game?
Data Data
Summary of profit grouped by solar system with profit numbers obfuscated.
— Get Profit per Station By Solar System including Region
SELECT SUM(profit), mapDenormalize.itemName, mapSolarSystems.solarSystemName, mapRegions.regionName, AVG(mapDenormalize.security)
FROM wallet
JOIN mapDenormalize
ON (wallet.stationID = mapDenormalize.itemID)
JOIN mapRegions
ON (mapDenormalize.regionID = mapRegions.regionID)
JOIN mapSolarSystems
ON (mapDenormalize.solarSystemID = mapSolarSystems.solarSystemID)
GROUP BY wallet.stationID
ORDER BY sum(profit) DESC
It turns out that Lowsec only accounts for 7% of our profits to date so I can’t complain about the nerf that is going to hit Lowsec capital builders given the upcoming compression changes.
Conclusion
The changes to compression are a welcomed change, even if it means retiring or heavily modifying the logistical chain for Lowsec capital production. I have a feeling that there are going to be more major changes in the next four upcoming Devblog posts. Stay tuned.
Increasing Faction Standings with Tags
Posted: 2012-07-10 Filed under: eveonline, industry, pos | Tags: amarr, brass, bronze, caldari, copper, crystal, csm7, data centers, diamond, electrum, faction, gallente, gold, highsec, Minmatar, palladium, platinum, pos, seleene, silver, tags 9 CommentsFor some time I have been queuing up production jobs that would be well suited for highsec. In order to anchor a POS in highsec, your corporation needs to have 5.0+ standings with the appropriate faction [detailed guide].
Since my main character has run a lot of Gallente missions in the past for ISK and has Social IV, he started off this task with a standing of 3.12 to the Gallente Federation.
Rather than grinding out more missions, I found out that there are a series of one-time agents for each faction that will trade in tags for standings. There are four Data Centers, one for each empire, located in New Eden where you can turn in tags.
You can collect these tags by running missions or by buying them off the market. A bunch of tags for one empire’s Data Center agents came out to around 352 M ISK on the Jita market. I didn’t see any major swings in prices or obvious market manipulation attempts, so I bought enough for all the agents.
Turning in these tags raised my standing from 3.12 to 4.95 with Social IV. After Social V completes, I should have more than the required 5.0 base standing in order to anchor a POS in highsec. [edit] Apparently Social skills do not come into play with anchoring rights. It appears I will have to do some more grinding to get the base standing up to 5.0+.
Chatting with all the Data Center agents
Since corporation standings are calculated as ‘the average of all corporation member standings towards that faction after the period of a week’, I will have to move this character into a 1 member corporation and wait for the standings to update.
No rush on this project, but I wanted to get the standings in order now so when it comes time to light up the production lines, I can immediately drop a POS. If you don’t want to raise your corporation’s standings yourself, you can even hire a guy to do it for you.
Perhaps I will wait for the POS rework before starting this production project. From what I have gathered listening to current CSM Chairman Seleene, the POS redesign is in the works. I haven’t been able to determine if this is slated for Winter 2012 or for Summer 2013.
k162space 