diff --git a/README.md b/README.md index 1d4d0ec..85d2773 100644 --- a/README.md +++ b/README.md @@ -1,35 +1,43 @@ # Pastebinner - -Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/pastebinner`. To experiment with that code, run `bin/console` for an interactive prompt. - -TODO: Delete this and the text above, and describe your gem - -## Installation - -Add this line to your application's Gemfile: - -```ruby -gem 'pastebinner' -``` - -And then execute: - - $ bundle - -Or install it yourself as: - - $ gem install pastebinner +Check out the examples folder for some examples. I will add more soon. ## Usage +Set these three environment variables: -TODO: Write usage instructions here +``` +pastebin_api_key +pastebin_username +pastebin_password +``` -## Development +If you want to scrape pastes, you can view a json response of the latest pastes by using the ```-s``` or ```--scrape_public``` options. +Creating pastes is built in, check ```lib/pastebinner.rb```, I am still working on adding in the functionality to use it on the command line. +The command line app can be used as follows: -After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment. +```shell +pastebinner --help +Usage: pastebinner [options] + -h, --help Show this help messae + -s, --scrape_public Scrape public pastes + -r, --raw Raw paste. Requires --key passed with a valid key + -g, --get_keys Get unique paste keys from public pastes + -k, --key= Unique paste key + -d, --download Download all public pastes to data directory + -t, --trending Trending pastes +``` +### Downloading pastes +Create a directory in the git repo named data, this is where the raw pastes will be stored. +Here is an example of what it looks like: -To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org). +```shell +pastebinner --download +Downloading paste data into the data directory... +Complete. +``` -## Contributing +The data directory will then be populated with pastebin raw paste files with the following naming scheme: -Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/pastebinner. +```shell +pastebin_paste_key_agiArDuG pastebin_paste_key_f9sZ4zfZ pastebin_paste_key_kkS48wHT pastebin_paste_key_rjxNDUbN pastebin_paste_key_vu5HSagJ pastebin_paste_key_ZZrUy4Fa +pastebin_paste_key_AHMT1YSC pastebin_paste_key_f9Zn6FWf pastebin_paste_key_kLVdVpTW pastebin_paste_key_rLDbHTZN pastebin_paste_key_vv6R1Dz4 +``` diff --git a/README.md.default b/README.md.default new file mode 100644 index 0000000..1d4d0ec --- /dev/null +++ b/README.md.default @@ -0,0 +1,35 @@ +# Pastebinner + +Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/pastebinner`. To experiment with that code, run `bin/console` for an interactive prompt. + +TODO: Delete this and the text above, and describe your gem + +## Installation + +Add this line to your application's Gemfile: + +```ruby +gem 'pastebinner' +``` + +And then execute: + + $ bundle + +Or install it yourself as: + + $ gem install pastebinner + +## Usage + +TODO: Write usage instructions here + +## Development + +After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment. + +To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org). + +## Contributing + +Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/pastebinner. diff --git a/bin/pastebinner b/bin/pastebinner index d78c955..806ba9c 100755 --- a/bin/pastebinner +++ b/bin/pastebinner @@ -10,8 +10,14 @@ pb = Pastebinner.new(ENV['pastebin_api_key'], ENV['pastebin_username'], ENV['pa def download_pastes(pb) pub_pastes = pb.scrape_public_pastes keys = pb.get_unique_paste_keys(pub_pastes) + data_dir = "../data/" + filename = "pastebin_paste_key" keys.map do |id| - File.write("../data/pastebin_paste_key_#{id[:key]}", pb.raw_paste_data(id[:key])) + if File.exist?(data_dir + filename) + puts "#{id[:key]} already exists on your filesystem, skipping..." + else + File.write(data_dir + filename + "_#{id[:key]}", pb.raw_paste_data(id[:key])) + end end end diff --git a/lib/configuration.rb b/lib/configuration.rb new file mode 100644 index 0000000..c37593d --- /dev/null +++ b/lib/configuration.rb @@ -0,0 +1,5 @@ +class Configuration +# this is going to be where i store config options like +# - database or directory. directory is built in, we just need to fine tune it a bit more +# - we will start doing just a standard sqlite file for raw pastes as a test +end diff --git a/lib/exceptions.rb b/lib/exceptions.rb index d5788a8..2ff5db8 100644 --- a/lib/exceptions.rb +++ b/lib/exceptions.rb @@ -1,5 +1,4 @@ class PastebinnerError < StandardError -end -raise PastebinnerError +end