spotmarket Graph Sample

What’s data without some graphs? Here is a small snip of the graphing that is currently in the spotmarket project.

Data

Jumps
2016-01-13 to 2016-06-07
17,802,640 Rows

Kill
2016-01-13 to 2016-06-07
9,819,421 Rows

Graphs

Jumps in Trade Hubs

2016-06-06_trade_hubs

Jumps and NPC Activity in The Northern Regions

2016-06-06_world_war_bee

Gurista Ratting Rates

2016-06-06_guristas

NPC Universe Overview

2016-06-06_universe_kills

Outer Ring

First pass at a map using D3.js inspired by HelicityBoson at http://www.machine9.net/?p=1111.

2016-06-06_outer_ring


spotmarket – 0.4

Overview

Whew, this was a big step in the right direction. I added a lot of sanity (keep everything in UTC), usability, and cleanup to the existing design. We’ve got moongoo, the beginnings of indexes, and CREST killmail data now flowing in to the growing dataset. The backend was improved by migrating to PostgreSQL 9.5.1 and I also cleaned up the install directions to try to make it easier.

0.4 Release

[bug] Change timestamp for zkillboard and markethistory consumer to use UTC for mental sanity
[enhancement] Add 404 error page
[enhancement] Change to postgres 9.5.1 to support jsonb
[enhancement] Parameterize graph functions
[enhancement] Change date format in graphs to ISO for mental sanity
[enhancement] Standardize table formatting
[enhancement] Supervisor to make Flask web service persistent
[enhancement] Add jsonb index on data.killmails for typeID and solarSystemID
[enhancement] Add paging to zKillboard consumer
[enhancement] Add check to resume from last recorded page for zKillboard consumer
[enhancement] Add basic exception handling to zKillboard consumer
[new] Creation of ship index report

 

2016-03-07_jumps2016-03-07_pirate_index2016-03-07_region_fountain2016-03-07_market_pilot_services

CREST Verified Moon Minerals

It is no secret that I have an affinity for finding accurate moon data. In a previous post, I’ve gone so far as to chart the regional density of each moon mineral, showing that there were built in skews as to where each type was placed for each class (R8, R16, etc).

In the Rubicon expansion, CCP introduced siphon units, which can be anchored near a POS to slowly leech items from within the shields. When an extractor is destroyed the killmail will show what moon mineral was in it. This, combined with the x,y,z location data that started to be exposed after the Parallax expansion, can be used to locate the nearest celestial object to give us a verified report of what moon mineral is contained in the object.

I gathered some examples for anyalisis, and over a long flight, I started to put some code together to use killmail x,y,z coordinates to parse CREST killmails. After implementation and testing, it has proven to be accurate so I’ve included the feature in this release. There is a manual step to get the moon mineral data into a table where it is used in the web frontend; I wanted to keep them separate to isolate the two datasets.

PostgreSQL Upgrade to 9.5.1

Rather than writing a parser for CREST killmails, I decided to store the JSON itself for simplicity. Developing on 9.3 with the json datatype proved to be a path that I did not want to go down. 9.3 is not the prefered version and lacked support for a fancy datatype inherit to 9.4+ and above. The following blog post also convinced me to migrate.

The proof is in the numbers so here’s a side-by-side comparison of the same query on the same dataset with only the datatype being different. The two virtual machines also had access to the same amount of CPU and RAM to control the results.

2016-03-13_postgres_9_5_performance_1

This query was proof enough that I made the right choice. Later that night while checking my logging table, I also noticed an improvement in insert speed.

9.3 (json) [kills] insert 2982 @ 93.81 rec/sec
9.5.1 (jsonb) [kills] insert 2850 @ 124.44 rec/sec

Not bad for about 45 minutes of testing and correcting install instructions.

0.5 Release

What’s up for the next release? Check out the TODO.md for a full list of items slated for each release. The main focus is going to be putting more control in the web front end, letting you enable/disable import items, add item to the market/zKillboard watch list, etc.


Trading 203: Mission Hub Stocking

Overview

This guide is indented to use a Highsec mission hub as a trading example.

I wanted to use a concrete example and disclose methods that I have developed over the past few years on how to seek out items to trade. Every trader has a technique, and I am no different. The following process is meant to guide you through the technique of selecting an item and gauging its profit potential.

Example

Product Source: Jita
Product Destination: Mission Hub of Dodixie
Weekday 02:00 Eve Time population: 349 Pilots
Item: Cybernetic Subprocessor – Basic

Item Selection

Every market is different so you need to due your due diligence.

For mission runners you might want to look at common mission ships such as the Raven or Dominix. Think about the entire package beyond the ship. Mission runners will need ammo, rigs, drones, and salvage equipment. In a PVP market, you will want to take a look at High Volume PVP Items.

For this trading example I picked a very common item. Almost all pilots have, at any given time, a Cybernetic Subprocessor of any grade in their head to improve training time.

The Market Window

The purpose of scanning the market window is to give you a ‘feel’ for the item.

Is obvious manipulation attempt obvious? Is the volume so low that you should not bother trading the item? Using this tool will help you weed out the error prone trades and focus your efforts.

I have a long list of items to browse through ranging from modules, ships, ammo, to even special high meta items. When you have 200 items to review, you should only be spending 5-10 seconds on each item to get a feel for the market. Quickly work down through your Quickbar in both regions, comparing results.

Market Window Using the Example

2012.12.13.02.32.15-combined

1. Price Trend

The Jita price has been rather stable over the past few months. Yes, there was a price increase from ~6.5 to around 8 M but remember that LP payouts were adjusted in a previous patch so I would not worry about the price raise to 8. Consider it a patch market blip.

The market in Dodixie is more volatile, as expected; a smaller group of traders is stocking the market and trying to make a profit. The range and period of the price waves don’t seem to bother me.

2. Volume

Rough guesstimation shows that around 50 move a day in Dodixie with larger spikes when traders are buying out the market to raise prices.

You can clearly see two major spikes in the Dodixie market where the price was ‘reset’ as I call it. This is normal behavior and I see it across the board in Highsec, Lowsec, and Nullsec. A larger trader will buyout all the items to a certain price threshold and then attempt to sell off stock at that newly established price. If your items were bought out at a profit, then you can ride the wave and restock at the new price hoping that it doesn’t crash back down (I’ll detail this process later).

3. Market Window Conclusion

Both the trends and volume pass the sanity check for me on this item. At this point I would look to use further tools to gauge profit potential.

Spreadsheet Time

Since our implant passes a sanity check, you can move on into more number work to see the item’s true potential.

1. Cut and paste a month worth of history from Eve into Excel. Ignore the most recent trading day as the numbers are not final and will throw off your calculations.

2. Do a find and replace on “ISK” to “” to get that extraneous text out of your sheet.

3. Adjust column size and add column headers. You should have something that looks like this.

2012-12-12_cybernetic_basic-1

Getting Useful Data

1. If you SUM the Quantity column, you can get a view into how many move in the region in a month.

For this implant we are are looking at 1,702 items/month. This figure will help you estimate how much you should buy at your source system. Based on your scale, you can buy a full month of items or small increments as liquidity allows.

Since a full month of implants to stock this region would come out to 13.694 B at Jita sell prices, you will opt for small portions.

2. Gauging profit potential is rather easy. Take the current Jita price and the last average price you have in Dodixie. Use the movement of the region of the month

(Jita Price – Dodixie Price) * Movement/Month = Region Movement/Month

2012-12-12_cybernetic_basic-2

3. The region potential doesn’t show you the volume in the system that will be your focus. For this you look at the Market Data tab to see where the orders are located.

2012.12.13.02.33.29-1

We can see that the majority are concentrated in Dodixie with a smaller concentration located in Bourynes. This informs me that the majority of trades are going to be concentrated in the Dodixie IX – Moon 20 station. This is where you should setup shop.

Closing Notes

Hopefully my method will help you see how to operate and fine tune your operation. I haven’t touched on logistics or item diversification — I have to save topics for other posts.


ninja! ninja! salvage!

Well, xxDeath came into our WH again with covops and stealth bombers (this makes the third time), since we again have a nullsec connection to their space. It seems odd that we keep finding nullsec connections, and odder that every other connection is within 5 jumps of xxdeath territory. So, since there was no way in hell I was taking my 190 mil hulk out w/ dudes on the loose, I went to my alts.

Sergeant Scowlyface, the salvager alt, has completed his salvage training.(tech 2 afterburners, MWDs, salvagers, and destroyers). Unfortunately, morleena, my combat alt, had her skill training put on hold once I realized that my current Orca skills on herleena, another alt on the same account, gave me a grand total of 2 seconds off my cycle time. So, I’m training mining director to IV and picking up the warfare linking skillbook once we get an enterence.

Anyway, having a salvager in Dodixie, I asked politely in local if anyone needed their missions salvaged. After the missions runners and mission ninjas alike laughed at me, I started training astrometrics to learn about this ninja salvaging hoo-haa. Three hours later, I undocked with basic combat probes and an expanded launcher only to have a  covops frigate explode directly in front of me. Looting the wreck and docking immediately was exhilarating, and netted me a sister expanded probe launcher and five sister combat probes, among other things that I couldn’t use.

During the 15 minute wait time while the victim moaned about losing his ship and the killer moaned about losing the wreck and the pod, I fitted the scanner and probes to my imicus and watched a short video. Undocking, I warped to a safespot and launched probes, hit scan, and waited to see ships I could warp to.

Fifteen seconds later, the scanner gave me…no results at all. Rechecking the video, I was told to set my filter to battleships and command ships. So, I set my overview to just command ships and battleships. After a few more minutes of cursing, I asked in local for help and learned that I was supposed to set my scan filter, not my overview filter, to battleships and command ships. This done, I scanned down a Rattlesnake, got into my Catalyst, and warped to him.

I found myself at a warp gate, and went through. I found myself in a sea of yellow wrecks, and proceeded to salvage my heart out. By the tenth wreck, I was already irritated at how slow my catalyst was, so I docked, buying a tristan at the station I was docking in, along with three small salvage tackles, 3 overdrive injectors, an 1mn AB II, and two cap rechargers: total fitting cost: 6mil. Total ship cost: 200k.

I warped back to the gate to find the gate gone, so I redocked, got my imicus, and scanned a new dude down. BMed, warped to w/ tristan. The mission was Worlds Collide, and after I salvaged the initial room, I went to the second room and began salvaging.

Four wrecks later, a catalyst shows up and begins salvaging the wrecks. I immidiatley angle towards the wreck he’s tractoring, lock it, and arc past, salvaging it before he can. I was immidiatley awash with the feeling of out-manuvering someone with an extreme tactical advantage, and proceded to out-salvage him for the next five wrecks before leaving to help close some WHs in our home.

So, ninja salvaging. So easy, less than one mil SP chars can do it, assuming they know how to work the scanner and an afterburner. Not very profitable by WH standards (400 mil paycheck yesterday), but extremley profitable from a dude flying in a navitas’s standpoint.

The moral standard in Eve is somewhere below a toilet in a sewer in the mob-run part of Detroit, so don’t expect anything you don’t nail down to still be there in the morning. At heart, ninja salvaging is a pretty tame moral deviation, since half the time the runners don’t even care about the salvage and the other half, they’re so sick of ninja salvagers that they made a salvage alt. So, if I find a bunch of unattended wrecks, I’ll assume that they’re fair game.

No-one take me up on this, but it seems to me that if a jerk in a destroyer can kill a hulk, he should be able to kill a frigate before getting concorded. If people are really annoyed by salvagers, then using a destroyer to suicide gank them and getting paid, then using the salvage the ninja salvaged to pay for a new destroyer, seems like a niche someone could fill pretty easily.

No-one do that tho. It’s stressful enough when a BS rat starts shooting at my tankless frigate, someone in an anti-frigate ship shooting at me will ruin my  whole day.

Fly safe, as I probe you.

P.S. I’ve decided to name my third char on my main account “Indigo Montoya”, but have no idea what to have him do. My main is gallente BS and miner, I have a salvage alt and can’t think of a third role I want to pay right now.