65 lines
2.2 KiB
Markdown
65 lines
2.2 KiB
Markdown
# Pastebinner
|
|
Check out the examples folder for some examples. I will add more soon.
|
|
|
|
## Usage
|
|
### Configuration
|
|
|
|
### Docker
|
|
- build the image:
|
|
```shell
|
|
docker build -t pastebinner .
|
|
```
|
|
- run the image:
|
|
```shell
|
|
docker run -it --env-file .env --network="host" pastebinner
|
|
```
|
|
|
|
Set the following environment variables and source them:
|
|
|
|
```
|
|
pastebin_api_key
|
|
pastebin_username
|
|
pastebin_password
|
|
```
|
|
|
|
If you want to scrape pastes, you can view a json response of the latest pastes by using the ```-s``` or ```--scrape_public``` options.
|
|
Creating pastes is built in, check ```lib/api_client.rb```, I am still working on adding in the functionality to use it on the command line.
|
|
The command line app can be used as follows:
|
|
|
|
```shell
|
|
Usage: pastebinner [options]
|
|
-h, --help Show this help messae
|
|
-v, --verbose Verbose http output (WIP)
|
|
-s, --scrape_public Scrape public pastes
|
|
-r, --raw Raw paste. Requires --key passed with a valid key
|
|
-g, --get_keys Get unique paste keys from public pastes
|
|
-k, --key= Unique paste key
|
|
-d, --download Download all public pastes to data directory
|
|
-j, --json Download all public pastes as a json into data directory
|
|
-t, --trending Trending pastes
|
|
```
|
|
### Downloading pastes
|
|
Create a directory in the git repo named data, this is where the raw pastes will be stored.
|
|
Here is an example of what it looks like:
|
|
|
|
##### Raw pastes to files
|
|
```shell
|
|
pastebinner --download
|
|
Downloading paste data into the data directory...
|
|
Complete.
|
|
```
|
|
|
|
##### JSON file with paste_metadata and paste_text
|
|
```shell
|
|
pastebinner --json
|
|
Downloading paste data as a json into the data directory...
|
|
Complete.
|
|
```
|
|
The data directory will then be populated with pastebin raw paste files with the following naming scheme:
|
|
|
|
```pastebin_paste_key_agiArDuG.raw``` or ```pastebin_paste_key_agiArDuG.json```
|
|
### To Add:
|
|
- fulltext search of raw pastes via elastic search, mysql, postgres, sqlite, or some other method
|
|
- adding exceptions
|
|
- adding configuration file support
|
|
- adding rspec tests
|