booboy
1b6759952e
quick typo
2019-02-02 04:35:15 -06:00
booboy
79a139a0fc
added examples of how you can automate downloading pastes with the command line binary and some systemd service/timer file pairs
2019-02-02 04:28:04 -06:00
booboy
fafd8268c0
added the ability to download either raw pastes or the full metadata + pastetext in one json document
2019-02-02 04:17:17 -06:00
booboy
e580c572f7
removed old mapping from pastebinner and also adjusted download to include metadata and fulltext of paste in the json
2019-02-02 03:52:56 -06:00
booboy
bf64a2f60e
added method to es helper to delete index and get mappings
2019-02-02 03:41:39 -06:00
booboy
6f715c3f86
this mapping has to be right now please
2019-02-02 02:44:59 -06:00
booboy
7db91e6053
pretty sure this mapping is going to work
2019-02-02 02:25:50 -06:00
booboy
4a2f27d5f6
was getting errors for previous mapping when trying to auto increment index with post requests, hopefully this fixes that.
2019-02-02 00:37:05 -06:00
booboy
ee93b17576
made a mapping that worked and some more stuff to the elastic search helper
2019-02-01 23:49:35 -06:00
booboy
2a39616379
added methods to update mappings
2019-02-01 22:44:41 -06:00
booboy
50b052a9bd
made elastic search helper better and made pastebinner pack json
2019-02-01 22:19:36 -06:00
booboy
ba20e8dc41
fixed some things with hashing the paste method and sending data to es
2019-02-01 20:38:14 -06:00
booboy
107c1ff559
added some methods to better prepare data for elasticsearch
2019-01-29 03:51:36 -06:00
booboy
c72f8c9a5f
put files in pastebinner dir
2019-01-28 23:58:30 -06:00
booboy
cac1352895
building skeletons of exceptions
2019-01-06 20:50:05 -06:00
booboy
0acc26f90e
small update
2018-11-18 21:10:45 -06:00
booboy
319822fdad
almost there with the mappings for metadata
2018-11-18 06:44:45 -06:00
booboy
18c1bde0a3
made a bunch of hashes that merge to prepare pastes and metadata for ES. pastes seem to work, but metadata is still messed up. i think it is because of the mappings that i did in the elastic_search_helper.rb script
2018-11-18 06:04:51 -06:00
booboy
345f826600
made the json encoding store metadata of a paste and the fulltext
2018-11-17 22:15:36 -06:00
booboy
d713b509d2
fixed one character typo in ENDPOINTS constants for scraping raw paste metadata
2018-11-17 22:02:22 -06:00
booboy
fa0bd05b06
added an encode_json method to prep for db
2018-11-17 18:37:26 -06:00
booboy
bd6aec9542
an update to readme, update to i dont know anymore i am tired and have to go to sleep look at the diff
2018-11-10 04:47:41 -06:00
booboy
fa41c0cb7f
encode_json method and a 1 character change in the filename check
2018-11-10 04:31:32 -06:00
booboy
ef312d12cf
added a method that we will later use to serialize the raw paste into json so we can ship it to a DB
2018-11-10 04:27:25 -06:00
booboy
879a897770
just added some psuedocode for an exception and a config file method
2018-11-10 04:14:32 -06:00
booboy
e49e94e0e0
moved examples to its own directory
2018-11-09 23:04:27 -06:00
booboy
c4fde291e4
made an actual readme to explain how to download pastes
2018-11-09 22:53:09 -06:00
booboy
4c43f54e96
added an option_parser class to use with the command line binary
2018-11-09 01:51:21 -06:00
booboy
ebde6ca0f2
messed with the cmd line options, some work now
2018-11-04 02:59:25 -06:00
booboy
9d34d53372
fixed trends. one character fix
2018-11-04 02:39:52 -06:00
booboy
eae20ccd0c
started to work on the command line application
...
added
-- raw
-- scrape_public_pastes
-- list_trending (still broken though. need to fix the api call)
-- added a method to extract just the keys from public pastes
-- moved the previous tests to an examples.rb file
need to mess around with all the different ways to parse arguments for the
command line program.
2018-11-04 02:28:57 -06:00
booboy
092bbb252b
added all of the methods.
...
will need to go back through and fix the scraping api endpoints to use the execute_query private method
2018-11-03 01:57:23 -05:00
kenna-bmcdevitt
2be2d4419c
fixed naming conventions and set this up to begin working on command line calls
2018-08-22 15:03:42 -05:00
kenna-bmcdevitt
eaf6207f22
added a couple more api calls
2018-08-22 14:00:13 -05:00
kenna-bmcdevitt
9da7b16a6e
made execute_query a work on the methods inside of the PasteBinner class
2018-08-22 13:32:53 -05:00
booboy
94568018e3
made execute_query a private method
2018-08-19 20:22:48 -05:00
booboy
447dc1f003
added an api_post method, and a get_api_user_key method for retrieving a session key which is required for listing user pastes.
2018-08-16 20:31:34 -05:00
booboy
da6c789a5c
initial commit
2018-08-16 14:54:08 -05:00
