roadmapping

[serapath]

@todo

• Run the scraper locally and make sure it runs and stores all contracts
• Write a tutorial how to use the updated github scraper repository and publish and run it on digital ocean
• Run scraper server on digital ocean droplet and make sure it keeps on running
• replace mySQL with filesystem and store all smart contract source codes in sourcecodes/
• create REST API server with routes to check:
  1. how much space the sourcecodes/**/* folder is using
  2. how many contracts are stored in total (maybe: https://stackoverflow.com/a/22957193)
  3. respond with the result of ls ./sourcecodes
• share/publish scraped data
  • dat share the folder of stored contracts using the the dat cli
• datdot-tcir#4: scraper until presentation for EthIndia
• datdot-cir#7 AST/type/data analysis
• datdot-cir#5 p2p publish data archive
• datdot-cir#2 AST import statement analysis and content hash replacement
• datdot-tcir#3 index structure (e.g. hypertrie(s))
• check that scraper is continues to run
• once all content has been scraped, try adding a new one to see if the scraper receives the updates - if not, make it work :-)
• datdot-tcir#6 maintain always up-to-date list of proxy servers
• add additional API to server /dat to tell if dat share ... is still serving
  • if so it should curl https://<digitaloceanserver ip>/dat returns e.g. true
• also http://104.248.146.46:3001/list has a broken formatting and the output seems to be formatted with HTML but it woud be better to have raw text with no HTML tags at all
• also can you change the README.MD and delete the old content in it and instead write the scraper server address and document the API? :-)
  • http://104.248.146.46:3001
    • /dat - indicates true or false to tell whether dat is still active or not
    • /daturl - returns the dat://<address> which can be used to download all contract
    • /count - gives the total amount of contracts currently stored on the server
    • /size - shows how much disk space contracts occupy
    • /list - shows a list of all contracts on the server
• write a tutorial (with screencast) how to update the digital ocean droplet with that changed code and keep it running
• scrape source codes manually from URLs below to add them to the database once
  • contact @serapath for some additional sources to scrape
• e.g. typeverify#1: verify items
• datdot-tcir#3: hypertrie(s) structure
  • start with a small subset of contracts (...maybe 20 items)
  • create a modified verifier module to still be able to verify them
  • process all items and verify them once the small subset of 20 works and can be verified
• #5 store all items in dat and publish on DHT
  • test success by loading the published dat address in [beaker browser]
    (https://beakerbrowser.com/)
• make a repository for a pure dat based p2p database node which can run on DappNode
  • https://twitter.com/thegrifft/status/1160220722947510272
• proof of concept by creating a datdot-tcir instance on digital ocean to pin the items archive
• work out fully featured concept of scsen

---

maybe we even need multiple servers - one on each continent? :-)

We could either use:

• https://github.com/beakerbrowser/homebase/ (if we decide to use hyperdrive)
• or we could use the dat-sdk to create our own pinning service
  • (i think i might prefer that, but not sure yet)
• or we can use dat-store, see:
  • https://gitter.im/ethereum/play?at=5d374d4509580b7bbb92e047

  • I'd suggest creating a digital ocean droplet and then running dat-store on it

  • Also here's some docs on how to make it run as a service: https://github.com/datproject/dat-store#linux-system-d

  • Right now I can run dat-store add ./ in a folder on my Windows machine, and it'll automatically share it

  • Then I run dat-store add dat://whatever on my digital ocean droplet, and it keeps it online