data_importer/README.md

# data_importer

Import common security data such as CVE, CPE, and Security Advisories from various CNAs into a rails app with a postgresql db backend.

## Supported data models:
- `Cve` data from [cve_list](https://github.com/CVEProject/cvelist) github repo.
- `Cpe` data from [nvd](https://nvd.nist.gov/products/cpe) 2.2 format.
- `GithubPoc` data from [nomi-sec](https://github.com/nomi-sec/PoC-in-GitHub) github repo.
- `InthewildCveExploit` data from [inthewild.io](https://inthewild.io/api/exploited) exploited feed.
- `TrickestPocCve` data from [trickest](https://github.com/trickest/cve) github repo.
- `CvemonCve` data from [ARPSyndicate](https://raw.githubusercontent.com/ARPSyndicate/cvemon/main/data.json) github repo.

## Initial Setup

### Build container
`docker-compose build`

### Database creation and seeding initial data
```
docker-compose run web rake db:create
docker-compose run web rake db:migrate
docker-compose run web rake db:seed
```

### Launch Pry console
`docker-compose run web rails console`

### HTTP API
For now unauthenticated api over localhost:3000 until I put in some basic token auth. All response data is json rendered.

#### Cves
```
  get "/cves", to: "cves#index"
  get "/cves/:cve_id", to: "cves#show"
  get "/cves/years/:year", to: "cves#show_year"
```
#### Cpes
```
  get "/cpes", to: "cpes#index"
  get "/cpes/:id", to: "cpes#show"
```

#### GithubPocs
```
  get "/github_pocs", to: "github_pocs#index"
  get "/github_pocs/:id", to: "github_pocs#show"
  get "/github_pocs/cve/:cve_id", to: "github_pocs#show_for_cve"
  get "/github_pocs/years/:year", to: "github_pocs#show_year"
```

#### InthewildCveExploits
```
  get "/inthewild_cve_exploits", to: "inthewild_cve_exploits#index"
  get "/inthewild_cve_exploits/:cve_id", to: "inthewild_cve_exploits#show"
```

#### TrickestPocCves
```
  get "/trickest_poc_cves", to: "trickest_poc_cves#index"
  get "/trickest_poc_cves/:id", to: "trickest_poc_cves#show"
  get "/trickest_poc_cves/cve/:cve_id", to: "trickest_poc_cves#show_for_cve"
  get "/trickest_poc_cves/years/:year", to: "trickest_poc_cves#show_year"
```

#### CvemonCves
```
  get "/cvemon_cves", to: "cvemon_cves#index"
  get "/cvemon_cves/:id", to: "cvemon_cves#show"
  get "/cvemon_cves/cve/:cve_id", to: "cvemon_cves#show_for_cve"
  get "/cvemon_cves/years/:year", to: "cvemon_cves#show_year"
```