|
|
70b4342abc
|
removed those env vars in the docker file. they can be specified in an environment file in the docker run process instead. i will detail how to do that in the readme file
|
2019-04-06 01:03:20 -05:00 |
|
|
|
c4199d052b
|
added to gitignore and added the required environment vars in dockerfile
|
2019-04-06 00:59:19 -05:00 |
|
|
|
9a1b1121d8
|
added a config file example in the examples directory
|
2019-04-06 00:20:01 -05:00 |
|
|
|
b38a3842b7
|
configuration example complete
|
2019-04-06 00:03:01 -05:00 |
|
|
|
67bc00bb1d
|
trying to add in support for configure block. launching console giving me bug with pb object
|
2019-03-29 01:00:11 -05:00 |
|
|
|
332efbe9cb
|
working on a Docker version
|
2019-03-28 23:12:15 -05:00 |
|
booboy
|
894879792d
|
Merge branch 'master' of https://git.mcdevitt.tech/bpmcdevitt/pastebinner
|
2019-03-20 00:14:12 -05:00 |
|
booboy
|
fe6cce2fc4
|
added a create_mapping method that does not work and a create_mappings.json. i think its the json but ill keep messing with it.
|
2019-03-20 00:13:37 -05:00 |
|
booboy
|
3ec18f5851
|
added a method to toggle restclient log to stdout in console
|
2019-03-19 02:14:03 -05:00 |
|
booboy
|
238d770d5e
|
this is now namedspaced correctly. at least much better than it was before this
|
2019-02-07 01:10:20 -06:00 |
|
booboy
|
f8d790e6df
|
making a sidekiq helper and started on a method to read data from raw json file to ship to es
|
2019-02-06 01:25:35 -06:00 |
|
booboy
|
9b0e11bcd9
|
moved examples and added an example of sinatra running sidekiq to view the paste jobs easier in a web ui without needing all of rails
|
2019-02-06 00:21:56 -06:00 |
|
booboy
|
c80bd01875
|
forgot to change file path
|
2019-02-05 01:03:08 -06:00 |
|
booboy
|
697b0e7679
|
moved around files for better organization
|
2019-02-05 00:46:13 -06:00 |
|
booboy
|
6c0b401d23
|
added the type of config block i want to have. not ready but setup a skeleton to use with it
|
2019-02-05 00:30:50 -06:00 |
|
booboy
|
c3fe3a2bb7
|
added sidekiq and redis to gemspec
|
2019-02-04 23:13:50 -06:00 |
|
booboy
|
0415c64960
|
made a paste_max limit option so you can query how many pastes you want
|
2019-02-04 22:45:26 -06:00 |
|
booboy
|
8d781e2499
|
made a job to scrape pastes and send to es, i got a rate limit error from pastebin so i need to slow down the calls i think, its 501 calls to the, raw metadata file, raw data file, and scrape index
|
2019-02-04 03:49:06 -06:00 |
|
booboy
|
7d0f304d43
|
made it into a difference method instead.
|
2019-02-04 01:48:50 -06:00 |
|
booboy
|
9916bb5c9d
|
made method to grab differences in keys arrays
|
2019-02-04 01:43:15 -06:00 |
|
booboy
|
07e5c4ac03
|
added 250 as a value to the scrape public paste so we can get more pastes
|
2019-02-03 03:18:03 -06:00 |
|
booboy
|
32fd073bf7
|
removed old comments
|
2019-02-03 03:08:04 -06:00 |
|
booboy
|
dc5ca66549
|
yaml safe load way to go
|
2019-02-03 03:03:33 -06:00 |
|
booboy
|
e104b0b015
|
good additions, got nested object working by loading paste_metadata as an array of hash instead of a string
|
2019-02-03 02:09:51 -06:00 |
|
booboy
|
e43e63b98a
|
im bad with markdown, fixed formatting
|
2019-02-02 04:57:28 -06:00 |
|
booboy
|
2ba1bde3b4
|
typo
|
2019-02-02 04:55:46 -06:00 |
|
booboy
|
dacfc616c4
|
updated readme
|
2019-02-02 04:54:26 -06:00 |
|
booboy
|
1b6759952e
|
quick typo
|
2019-02-02 04:35:15 -06:00 |
|
booboy
|
79a139a0fc
|
added examples of how you can automate downloading pastes with the command line binary and some systemd service/timer file pairs
|
2019-02-02 04:28:04 -06:00 |
|
booboy
|
fafd8268c0
|
added the ability to download either raw pastes or the full metadata + pastetext in one json document
|
2019-02-02 04:17:17 -06:00 |
|
booboy
|
e580c572f7
|
removed old mapping from pastebinner and also adjusted download to include metadata and fulltext of paste in the json
|
2019-02-02 03:52:56 -06:00 |
|
booboy
|
bf64a2f60e
|
added method to es helper to delete index and get mappings
|
2019-02-02 03:41:39 -06:00 |
|
booboy
|
6f715c3f86
|
this mapping has to be right now please
|
2019-02-02 02:44:59 -06:00 |
|
booboy
|
7db91e6053
|
pretty sure this mapping is going to work
|
2019-02-02 02:25:50 -06:00 |
|
booboy
|
4a2f27d5f6
|
was getting errors for previous mapping when trying to auto increment index with post requests, hopefully this fixes that.
|
2019-02-02 00:37:05 -06:00 |
|
booboy
|
ee93b17576
|
made a mapping that worked and some more stuff to the elastic search helper
|
2019-02-01 23:49:35 -06:00 |
|
booboy
|
2a39616379
|
added methods to update mappings
|
2019-02-01 22:44:41 -06:00 |
|
booboy
|
50b052a9bd
|
made elastic search helper better and made pastebinner pack json
|
2019-02-01 22:19:36 -06:00 |
|
booboy
|
ba20e8dc41
|
fixed some things with hashing the paste method and sending data to es
|
2019-02-01 20:38:14 -06:00 |
|
booboy
|
107c1ff559
|
added some methods to better prepare data for elasticsearch
|
2019-01-29 03:51:36 -06:00 |
|
booboy
|
c72f8c9a5f
|
put files in pastebinner dir
|
2019-01-28 23:58:30 -06:00 |
|
booboy
|
cac1352895
|
building skeletons of exceptions
|
2019-01-06 20:50:05 -06:00 |
|
booboy
|
9a4f1251dd
|
fixed an issue that was occurring with downloading the raw pastes
|
2019-01-06 20:21:35 -06:00 |
|
booboy
|
fc82bd78d1
|
Merge branch 'master' of https://git.mcdevitt.tech/bpmcdevitt/pastebinner
|
2019-01-06 19:26:49 -06:00 |
|
booboy
|
46526a21e9
|
made pry console startup with just a pb object for easy experimentation with the methods
|
2019-01-06 19:26:16 -06:00 |
|
booboy
|
0acc26f90e
|
small update
|
2018-11-18 21:10:45 -06:00 |
|
booboy
|
319822fdad
|
almost there with the mappings for metadata
|
2018-11-18 06:44:45 -06:00 |
|
booboy
|
18c1bde0a3
|
made a bunch of hashes that merge to prepare pastes and metadata for ES. pastes seem to work, but metadata is still messed up. i think it is because of the mappings that i did in the elastic_search_helper.rb script
|
2018-11-18 06:04:51 -06:00 |
|
booboy
|
345f826600
|
made the json encoding store metadata of a paste and the fulltext
|
2018-11-17 22:15:36 -06:00 |
|
booboy
|
d713b509d2
|
fixed one character typo in ENDPOINTS constants for scraping raw paste metadata
|
2018-11-17 22:02:22 -06:00 |
|
