rails app that takes and stores pastes from pastebin
Find a file
2019-02-03 03:03:33 -06:00
bin good additions, got nested object working by loading paste_metadata as an array of hash instead of a string 2019-02-03 02:09:51 -06:00
lib yaml safe load way to go 2019-02-03 03:03:33 -06:00
spec made elastic search helper better and made pastebinner pack json 2019-02-01 22:19:36 -06:00
.gitignore wrote a method to download all the raw pastes to a file inside of a directory. 2018-11-05 03:15:29 -06:00
.rspec initial commit 2018-08-16 14:54:08 -05:00
.travis.yml initial commit 2018-08-16 14:54:08 -05:00
Gemfile made elastic search helper better and made pastebinner pack json 2019-02-01 22:19:36 -06:00
LICENSE Initial commit 2018-08-16 14:52:20 -05:00
pastebinner.gemspec made elastic search helper better and made pastebinner pack json 2019-02-01 22:19:36 -06:00
Rakefile made elastic search helper better and made pastebinner pack json 2019-02-01 22:19:36 -06:00
README.md im bad with markdown, fixed formatting 2019-02-02 04:57:28 -06:00
README.md.default made an actual readme to explain how to download pastes 2018-11-09 22:53:09 -06:00

Pastebinner

Check out the examples folder for some examples. I will add more soon.

Usage

Set the following environment variables:

pastebin_api_key
pastebin_username
pastebin_password

If you want to scrape pastes, you can view a json response of the latest pastes by using the -s or --scrape_public options. Creating pastes is built in, check lib/pastebinner.rb, I am still working on adding in the functionality to use it on the command line. The command line app can be used as follows:

Usage: pastebinner [options]
    -h, --help                       Show this help messae
    -v, --verbose                    Verbose http output (WIP)
    -s, --scrape_public              Scrape public pastes
    -r, --raw                        Raw paste. Requires --key passed with a valid key
    -g, --get_keys                   Get unique paste keys from public pastes
    -k, --key=                       Unique paste key
    -d, --download                   Download all public pastes to data directory
    -j, --json                       Download all public pastes as a json into data directory
    -t, --trending                   Trending pastes

Downloading pastes

Create a directory in the git repo named data, this is where the raw pastes will be stored. Here is an example of what it looks like:

Raw pastes to files
pastebinner --download
Downloading paste data into the data directory...
Complete.
JSON file with paste_metadata and paste_text
pastebinner --json
Downloading paste data as a json into the data directory...
Complete.

The data directory will then be populated with pastebin raw paste files with the following naming scheme:

pastebin_paste_key_agiArDuG.raw or pastebin_paste_key_agiArDuG.json

To Add:

  • fulltext search of raw pastes via elastic search, mysql, postgres, sqlite, or some other method
  • adding exceptions
  • adding configuration file support
  • adding rspec tests