mirror of
https://github.com/kennethreitz/coinbin.org.git
synced 2026-07-22 05:09:31 +00:00
Claude
678722d4d4
Brings the long-dead service back to a bootable state. Addresses the modernization epic (#30) and its work items: - #31 scraper: replace the removed CoinMarketCap `views/all` HTML scrape with a JSON price API (CoinGecko by default, no key required). Keeps the get_coins() return shape and MWT cache; handles non-unique symbols by keeping the highest-market-cap entry. - #32 runtime: drop the removed `gaiohttp` gunicorn worker / `aiohttp<2.0`; Procfile now uses `gthread`. - #33 deps/runtime: migrate off unmaintained libs — flask-cache -> flask-caching, drop flask-common, graphene 1/2 -> graphene 3, raven -> sentry-sdk, Python 3.6 -> 3.11+. Replace flask-graphql (incompatible with graphene 3) with a direct schema.execute() view. Add a JSON provider so Flask 3 can serialize Decimal. Make DB/forecast imports lazy so the app boots without them; only force HTTPS outside DEBUG. - #34 ingestion: add ingest.py worker and committed schema.sql for the api_coin price-history table. - #35 forecasting: migrate fbprophet -> prophet behind a FORECASTS_ENABLED flag with lazy, optional imports. Dependency management moved to uv: pyproject.toml + uv.lock, replacing the 2017-era Pipfile/Pipfile.lock. README and app.json updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UMDxE5JVcuzpKyiHercWaa
56 lines
1.6 KiB
Python
56 lines
1.6 KiB
Python
"""Price-history ingestion worker.
|
|
|
|
Fetches current prices via scraper.get_coins() and appends them to the
|
|
`api_coin` table that powers /history and /forecast. Run it on a schedule
|
|
(e.g. Heroku Scheduler, cron, or a clock dyno):
|
|
|
|
python ingest.py
|
|
|
|
Set DATABASE_URL to the target Postgres instance. Set INGEST_PRO_URL (or
|
|
HEROKU_POSTGRESQL_TEAL_URL) to also mirror into the "pro" database. The
|
|
`api_coin` schema is created from schema.sql if it does not already exist.
|
|
"""
|
|
|
|
import os
|
|
import datetime
|
|
|
|
import records
|
|
|
|
from scraper import get_coins
|
|
|
|
|
|
def _ensure_schema(db):
|
|
here = os.path.dirname(os.path.abspath(__file__))
|
|
with open(os.path.join(here, 'schema.sql')) as f:
|
|
db.query(f.read())
|
|
|
|
|
|
def _insert_coins(db, coins, when):
|
|
insert = "INSERT INTO api_coin (name, value, date) VALUES (:name, :value, :date)"
|
|
for coin in coins.values():
|
|
try:
|
|
value = float(coin['usd'])
|
|
except (TypeError, ValueError):
|
|
continue
|
|
db.query(insert, name=coin['name'], value=value, date=when)
|
|
|
|
|
|
def ingest():
|
|
when = datetime.datetime.now(datetime.timezone.utc)
|
|
coins = get_coins()
|
|
|
|
targets = [os.environ.get('DATABASE_URL')]
|
|
pro_url = os.environ.get('INGEST_PRO_URL') or os.environ.get('HEROKU_POSTGRESQL_TEAL_URL')
|
|
if pro_url:
|
|
targets.append(pro_url)
|
|
|
|
for url in targets:
|
|
db = records.Database(url) if url else records.Database()
|
|
_ensure_schema(db)
|
|
_insert_coins(db, coins, when)
|
|
print('Ingested {} coins at {}'.format(len(coins), when.isoformat()))
|
|
|
|
|
|
if __name__ == '__main__':
|
|
ingest()
|