boogerlad/mj: backend/scrape

boogerlad

typo

on December 16, 2020

albertacannabis.org.js

fill in the alberta cannabis stub

on September 28, 2020
anticaptcha.js

scraper stub of albertcannabis.org

on February 27, 2020
bccannabisstores.com.js

don't attempt to insert into db if stock < 0

on September 28, 2020
common.js

puppeteer can complain about zero width screenshots

on September 29, 2020
dc.js

constraint for store times to be minutes within a week store_open_exception table for things like holidays commonize store name sanitization and geocoding/timezone/osrm hints rebrand summit cannabis dc properly update create_store to work with commonized functions, new times table remove duplicated `require`s

on August 10, 2020
image.sh

fix updating product_image dimensions. counted parameters incorrectly :( if it's time to update the images on cloudinary, the images on cloudinary should be updated if it does exist(200) or created if does not exist(404)

on August 13, 2020
ocs.ca.js

ocs continues to surprise with stupid non ascii in url

on November 26, 2020
stores.js

the store table has addresses parsed by google. for the unique constraint to work on insert, parsed addresses are required. the store scraper naively attempts to insert every store. this works, but is expensive(~10usd). introduce store_temp table which contains the raw addresses from the previous scrape to insert only new stores and show removed stores. should probably truncate store_temp once a month to update longitude, latitude and open times

on August 24, 2020