pastebinner/README.md

# Pastebinner
Check out the examples folder for some examples. I will add more soon.

## Configuration

#### Docker 
- Build the image: 
```shell 
docker build -t pastebinner .
```
- Run the image:
```shell
docker run -it --env-file .env --network="host" pastebinner
```
#### Console

The console will launch you into a running pry session where you will have access to an elasticsearch object and a pastebinner object. From there you can poke around and investigate further how the program works.

I am working on setting it up with docker-compose to connect elasticsearch and redis in docker containers. 
Set the following environment variables in a file named .env and source them:

```
pastebin_api_key
pastebin_username
pastebin_password
```

optional environment variables if planning on using elasticsearch and redis
```
elasticsearch_url
redis_url
```


If you want to scrape pastes, you can view a json response of the latest pastes by using the ```-s``` or ```--scrape_public``` options.
Creating pastes is built in, check ```lib/api_client```

#### Commandline
The command line app can be used as follows:

```shell
Usage: pastebinner [options]
    -h, --help                       Show this help messae
    -v, --verbose                    Verbose http output (WIP)
    -s, --scrape_public              Scrape public pastes
    -r, --raw                        Raw paste. Requires --key passed with a valid key
    -g, --get_keys                   Get unique paste keys from public pastes
    -k, --key=                       Unique paste key
    -d, --download                   Download all public pastes to data directory
    -j, --json                       Download all public pastes as a json into data directory
    -t, --trending                   Trending pastes
```
### Downloading pastes
Create a directory in the git repo named data, this is where the raw pastes will be stored.
Here is an example of what it looks like:

##### Raw pastes to files
```shell
pastebinner --download
Downloading paste data into the data directory...
Complete.
```

##### JSON file with paste_metadata and paste_text
```shell
pastebinner --json
Downloading paste data as a json into the data directory...
Complete.
```
The data directory will then be populated with pastebin raw paste files with the following naming scheme:

```pastebin_paste_key_agiArDuG.raw``` or ```pastebin_paste_key_agiArDuG.json```
### To Add:
- fulltext search of raw pastes via elastic search, mysql, postgres, sqlite, or some other method
- adding exceptions
- adding configuration file support
- adding rspec tests