bpmcdevitt/data_importer
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

add more verbosity

Browse source
This commit is contained in:
Brendan McDevitt 2022-04-06 02:45:11 -05:00
parent 4ae8f9e078
commit d81c31febe
2 changed files with 3 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
README.md
View file

@ -1,6 +1,6 @@
# data_importer # data_importer
Import common security data such as CVE, CPE, and Advisories from various CNAs into a rails rails app with a postgresql db backend. Import common security data such as CVE, CPE, and Security Advisories from various CNAs into a rails app with a postgresql db backend.
## Supported data models: ## Supported data models:
- `Cve` data from [cve_list](https://github.com/CVEProject/cvelist) mitre. - `Cve` data from [cve_list](https://github.com/CVEProject/cvelist) mitre.
@ -24,6 +24,7 @@ docker-compose run web rake db:seed
`docker-compose run web rails console` `docker-compose run web rails console`
### HTTP API ### HTTP API
For now unauthenticated api over localhost:3000 until I put in some basic token auth. All response data is json rendered.
#### Cves #### Cves
``` ```

1
lib/cpe_importer.rb
View file

@ -49,6 +49,7 @@ class CpeImporter
end end
def self.import(bulk_count = 20000, filepath = '/data_importer/data/official-cpe-dictionary_v2.2.xml.gz') def self.import(bulk_count = 20000, filepath = '/data_importer/data/official-cpe-dictionary_v2.2.xml.gz')
puts "Now importing Cpes."
Zlib::GzipReader.open(filepath) do |file| Zlib::GzipReader.open(filepath) do |file|
items = [] items = []
Nokogiri::XML::Reader.from_io(file).each do |node| Nokogiri::XML::Reader.from_io(file).each do |node|