data_importer

bpmcdevitt/data_importer

Fork 0

Commit graph

14b167e255 enforce utf-8 some more main kenna-bmcdevitt 2024-08-22 14:00:44 -05:00
3b517dba43 force utf8 encoding kenna-bmcdevitt 2024-08-21 12:24:35 -05:00
d6ff9fe367 more updates kenna-bmcdevitt 2024-08-21 10:30:18 -05:00
e675f9c2ac updated app kenna-bmcdevitt 2024-08-20 14:37:22 -05:00
e3931d3bb1 got working poc code for oval parsing oracle linux advisories. keep it up Brendan McDevitt 2022-05-30 21:38:20 -05:00
4868ef67d9 begin setting up experimenting with oval parser Brendan McDevitt 2022-05-30 12:18:40 -05:00
1221d76340 added dahua technolgies captured data potential from advisory example Brendan McDevitt 2022-05-22 18:42:29 -05:00
5639928760 added csw to captrued data example Brendan McDevitt 2022-05-22 18:23:57 -05:00
428f42b666 fix typo Brendan McDevitt 2022-05-21 10:50:01 -05:00
95dfbe533c added more notes on cisas advisory format Brendan McDevitt 2022-05-21 10:16:23 -05:00
70985c7511 added checkpoint captured data potential from advisory example Brendan McDevitt 2022-05-21 10:12:08 -05:00
e5cea3869f added cert@vde captured data potential from advisory example Brendan McDevitt 2022-05-21 10:02:42 -05:00
3531427dad add notes for cert/cc Brendan McDevitt 2022-05-20 23:30:23 -05:00
ffb63d3122 added brocade communication systems captured data potential from advisory example Brendan McDevitt 2022-05-20 23:27:11 -05:00
8797fdbd8d add blackberry to captured data potential from advisory example Brendan McDevitt 2022-05-20 23:21:30 -05:00
992014eb77 bitdefener adivsory site 404ing Brendan McDevitt 2022-05-20 23:08:09 -05:00
c581f850f0 added becton, dickison and company example captured data potential from advisory Brendan McDevitt 2022-05-20 20:24:39 -05:00
1cb04b140f added avaya captured data potential example Brendan McDevitt 2022-05-20 17:53:05 -05:00
c4901695d9 added example captured data potential from advisory for autodesk Brendan McDevitt 2022-05-20 17:28:56 -05:00
9d0358e866 1 char fix Brendan McDevitt 2022-05-20 17:22:11 -05:00
c1b3ed8fa1 add cna 7 jira captured potential from advisory Brendan McDevitt 2022-05-20 17:21:27 -05:00
955e7fd5fc fix cna 6 example and wrap it in quote for markdown Brendan McDevitt 2022-05-20 16:25:35 -05:00
31d8c575df added arista networks cna 6 example captured data potential from advisory Brendan McDevitt 2022-05-20 16:24:46 -05:00
6502ad71cd added apple cna 5 example captured data potential from advisory Brendan McDevitt 2022-05-20 16:09:06 -05:00
d9fe9d3adb added more notes on the android/google bulletins Brendan McDevitt 2022-05-20 15:59:45 -05:00
2d101e6988 added ampere cna 3 example captured data potential from advisory Brendan McDevitt 2022-05-20 15:56:26 -05:00
090fe2a119 added amd cna 2 example captured data potential from advisory Brendan McDevitt 2022-05-20 15:51:35 -05:00
222619e03a added adobe cna 1 example captured data potential from advisory Brendan McDevitt 2022-05-20 15:37:53 -05:00
2825e1105d showcase example of captured data potential. this is what we will fill for each CNA Brendan McDevitt 2022-05-20 15:27:16 -05:00
c3732be9e7 cleanup the markdown document for html with advisories and add potential what we can capture. need to backfill for cnas 1-40 Brendan McDevitt 2022-05-20 15:24:26 -05:00
20341377a3 added cna 41 Brendan McDevitt 2022-05-20 13:55:11 -05:00
24017c41d4 added cna 40 Brendan McDevitt 2022-05-20 11:54:36 -05:00
bd831220c5 added cna 39 Brendan McDevitt 2022-05-20 11:51:41 -05:00
0f39558b55 added cna 38 Brendan McDevitt 2022-05-20 11:49:58 -05:00
2e26f79ca0 added cna 37 Brendan McDevitt 2022-05-20 11:46:30 -05:00
8cfec21541 add cna 36 Brendan McDevitt 2022-05-20 11:44:27 -05:00
57b6287a23 cna data fixed on upstream end Brendan McDevitt 2022-05-20 10:48:37 -05:00
a659d5f45f added cna 35 Brendan McDevitt 2022-05-11 23:35:59 -05:00
91d57f5b5a added cna 34 Brendan McDevitt 2022-05-11 23:33:41 -05:00
9b8460fd5d added cna 33 Brendan McDevitt 2022-05-11 23:30:55 -05:00
9ca0a99613 added cna 32 Brendan McDevitt 2022-05-11 23:29:05 -05:00
8c4bbf9e88 added cna 31 Brendan McDevitt 2022-05-11 13:19:02 -05:00
b4b4d77256 make cnas_with_apis.md use same as other mds Brendan McDevitt 2022-05-11 12:40:22 -05:00
b98c79f4fe revert it nevermind i think ill do one markdown doc per cna Brendan McDevitt 2022-05-05 01:00:06 -05:00
efda2ffeaf put these in a subdir to organize it better Brendan McDevitt 2022-05-05 00:57:21 -05:00
75b9b4e9b9 Revert "add empty dir one for each shortname cna so we can get more detailed notes made on each" Brendan McDevitt 2022-05-05 00:48:11 -05:00
998d7c085a add empty dir one for each shortname cna so we can get more detailed notes made on each Brendan McDevitt 2022-05-05 00:39:33 -05:00
c9551eaa7b remove embed Brendan McDevitt 2022-05-05 00:24:18 -05:00
24993761a4 embed the image Brendan McDevitt 2022-05-05 00:21:10 -05:00
8a3ccd7a1a test image inline Brendan McDevitt 2022-05-05 00:19:47 -05:00
2361868ff6 moved cna_readme_notes and all files in it to doc dir in top root level of project Brendan McDevitt 2022-05-05 00:13:33 -05:00
ebd375fa61 fix the headings Brendan McDevitt 2022-05-05 00:01:57 -05:00
b5601f375a make other format advisory cna doc look better Brendan McDevitt 2022-05-04 23:59:19 -05:00
58e3ca234e make html advisory cna doc look better Brendan McDevitt 2022-05-04 23:56:58 -05:00
fef1088ccc adjust format of cna with no advisories doc Brendan McDevitt 2022-05-04 23:52:16 -05:00
d587c9131a trying to format the md doc better2 Brendan McDevitt 2022-05-04 23:48:49 -05:00
adfe62353b trying to format the md doc better Brendan McDevitt 2022-05-04 23:47:27 -05:00
9865c276b6 added cisa cna Brendan McDevitt 2022-05-04 23:41:52 -05:00
4e982c965c added cybellum cna Brendan McDevitt 2022-05-04 23:40:18 -05:00
d8388d3bbd added crafter cms Brendan McDevitt 2022-05-04 23:35:49 -05:00
8085a17e98 added cloudflare cna Brendan McDevitt 2022-05-04 23:33:00 -05:00
f7ec0bd58b added cisco cna Brendan McDevitt 2022-05-04 23:27:46 -05:00
a473905459 added chrome CNA Brendan McDevitt 2022-05-04 23:25:11 -05:00
2e46e622e4 add checkpoint cna Brendan McDevitt 2022-05-04 23:22:23 -05:00
befa32ee5f added cert@vde cna Brendan McDevitt 2022-05-04 22:33:40 -05:00
83da136330 more cnas Brendan McDevitt 2022-05-04 22:28:10 -05:00
232188a8c7 add second link for adobe advisories Brendan McDevitt 2022-05-04 20:10:55 -05:00
884c1186da typo in airbus entry Brendan McDevitt 2022-05-04 20:01:51 -05:00
3b43b23156 update cnas with apis Brendan McDevitt 2022-05-04 20:00:56 -05:00
8ace150878 make it more markdown Brendan McDevitt 2022-05-04 19:59:54 -05:00
efb8feac79 20 cnas down today. Brendan McDevitt 2022-05-04 19:53:33 -05:00
950035d3f3 left off at cna 18 Brendan McDevitt 2022-05-04 13:13:50 -05:00
c62d191ddc added a cna_readme_notes with some documents about how each cna records their security advisories Brendan McDevitt 2022-05-04 12:45:15 -05:00
5899e5c14d got a pretty solid summary scraper working. need to still clean all of it up. keep testing against all of the index urls to make this go faster and more efficient Brendan McDevitt 2022-05-04 01:00:46 -05:00
05c5c87808 beginnings of the advisory hash Brendan McDevitt 2022-04-28 14:37:47 -05:00
9925e0d28b starting to add in cna scrapers. first one is adobe. wrote up to index_hash method that allows me to pull a hash of all urls for each advisory id Brendan McDevitt 2022-04-28 13:38:05 -05:00
3a32d6086f add route Brendan McDevitt 2022-04-27 01:31:57 -05:00
763de4b83d added some methods for querying the vuln data Brendan McDevitt 2022-04-27 01:20:40 -05:00
01a9373cfd added route for cisa known exploits Brendan McDevitt 2022-04-27 01:11:36 -05:00
db1ef2e01a added support for cisa known exploits json feed Brendan McDevitt 2022-04-26 23:56:35 -05:00
fe415b0c13 add more upsert_all Brendan McDevitt 2022-04-25 12:58:39 -05:00
3c6828f0fd make trickest use upsert_all Brendan McDevitt 2022-04-22 12:12:45 -05:00
8261a53800 make poc in github importer use upsert_all Brendan McDevitt 2022-04-22 01:48:49 -05:00
ddecc2820c now we work with ruby:latest docker image Brendan McDevitt 2022-04-21 14:30:29 -05:00
c7e3bf4fb7 fix cpe importer worker using non existant import method. we need donwload_and_import method instead Brendan McDevitt 2022-04-20 19:05:44 -05:00
f55a21439e add route for gsd Brendan McDevitt 2022-04-20 13:22:22 -05:00
53aa79442e add route for gsds Brendan McDevitt 2022-04-20 00:22:45 -05:00
794648e0dd added gsd importer Brendan McDevitt 2022-04-20 00:15:01 -05:00
ef3a454a26 add timestamps to github advisories so upsert_all works the best Brendan McDevitt 2022-04-19 14:50:18 -05:00
9154a9cb9c make github advisory importer use upsert_all Brendan McDevitt 2022-04-19 14:22:59 -05:00
a2fa2fbb62 fix github advisory crontab entry Brendan McDevitt 2022-04-19 13:52:47 -05:00
3d6c30b8a1 adjust cve list importer simplify since we are using upsert_all Brendan McDevitt 2022-04-19 10:46:58 -05:00
2b28810cf2 rubocop commit Brendan McDevitt 2022-04-19 02:37:27 -05:00
c4240a4b2e typo Brendan McDevitt 2022-04-19 02:09:49 -05:00
1f8de20b88 fix poc in github job Brendan McDevitt 2022-04-19 02:02:07 -05:00
d3348c2ea2 edit README. add faktory stuff to README Brendan McDevitt 2022-04-19 01:26:14 -05:00
91d7c0f499 made basic skeleton for crontab of all workers we have. lets see how it goes then we can tweak from there Brendan McDevitt 2022-04-19 01:10:56 -05:00
59fba273f6 added trickest_poc_cve_importer_worker Brendan McDevitt 2022-04-19 00:36:01 -05:00
2f28ebae73 added poc_in_github_importer_worker Brendan McDevitt 2022-04-19 00:34:19 -05:00
785a02b53c added inthewild_cve_exploit_importer_worker Brendan McDevitt 2022-04-19 00:32:40 -05:00