bpmcdevitt/data_importer

Fork 0

This is a rails/postgres application that will serve json data from the following data sources: - Cves - Cpes - CNA security advisories - GHSA Github security advisories - Github repositories that track public exploits for cves. - A list of github usernames github API data.

Find a file

Brendan McDevitt 4ae8f9e078 update README		2022-04-06 02:35:02 -05:00
app	small adjustment to the way the inthewild_cve_exploits earliest_report data is stored	2022-04-06 02:21:18 -05:00
bin	make json rendering for cpes and cves	2022-04-04 13:18:03 -05:00
config	small adjustment to the way the inthewild_cve_exploits earliest_report data is stored	2022-04-06 02:21:18 -05:00
db	update README	2022-04-06 02:35:02 -05:00
lib	added a way to pull exploit feed info from inthewild.io exploits feed	2022-04-06 02:08:19 -05:00
log	init commit rails new data_importer	2022-03-30 22:12:56 -05:00
public	init commit rails new data_importer	2022-03-30 22:12:56 -05:00
storage	init commit rails new data_importer	2022-03-30 22:12:56 -05:00
test	make json rendering for cpes and cves	2022-04-04 13:18:03 -05:00
tmp	init commit rails new data_importer	2022-03-30 22:12:56 -05:00
vendor	init commit rails new data_importer	2022-03-30 22:12:56 -05:00
.gitignore	init commit rails new data_importer	2022-03-30 22:12:56 -05:00
.ruby-version	init commit rails new data_importer	2022-03-30 22:12:56 -05:00
config.ru	init commit rails new data_importer	2022-03-30 22:12:56 -05:00
docker-compose.yml	docker-compose and postgres basic rails app skeleton	2022-03-30 22:52:39 -05:00
Dockerfile	added files for importing cves from cvelist repo to database	2022-04-01 13:36:20 -05:00
entrypoint.sh	docker-compose and postgres basic rails app skeleton	2022-03-30 22:52:39 -05:00
Gemfile	make json rendering for cpes and cves	2022-04-04 13:18:03 -05:00
Gemfile.lock	make json rendering for cpes and cves	2022-04-04 13:18:03 -05:00
package.json	init commit rails new data_importer	2022-03-30 22:12:56 -05:00
Rakefile	init commit rails new data_importer	2022-03-30 22:12:56 -05:00
README.md	update README	2022-04-06 02:35:02 -05:00

README.md

data_importer

Import common security data such as CVE, CPE, and Advisories from various CNAs into a rails rails app with a postgresql db backend.

Supported data models:

Cve data from cve_list mitre.
Cpe data from nvd 2.2 format.
GithubPoc data from nomi-sec.
InthewildCveExploit data from inthewild.io exploited feed.

Initial Setup

Build container

docker-compose build

Database creation and seeding initial data

docker-compose run web rake db:create
docker-compose run web rake db:migrate
docker-compose run web rake db:seed

Launch Pry console

docker-compose run web rails console

HTTP API

Cves

  get "/cves", to: "cves#index"
  get "/cves/:cve_id", to: "cves#show"
  get "/cves/years/:year", to: "cves#show_year"

Cpes

  get "/cpes", to: "cpes#index"
  get "/cpes/:id", to: "cpes#show"

GithubPocs

  get "/github_pocs", to: "github_pocs#index"
  get "/github_pocs/:id", to: "github_pocs#show"
  get "/github_pocs/cve/:cve_id", to: "github_pocs#show_for_cve"

InthewildCveExploits

  get "/inthewild_cve_exploits", to: "inthewild_cve_exploits#index"
  get "/inthewild_cve_exploits/:cve_id", to: "inthewild_cve_exploits#show"