o
h
@ s d dl Z d dlZd dlZd dlmZ d dlmZ d dlmZ ee
j d Z
ee
d d dlmZ d dlmZ d d lmZmZmZmZmZ d d
lmZ d d ed ed
ededB dedB def ddZde de de de fddZ de de de de fddZ dd Z!e"d kre! dS dS ) N)Path) load_dotenv)escape z.env) crawl_motie)
crawl_moef) has_motie_idinsert_motie_id has_moef_idinsert_moef_idget_active_chat_ids)broadcast_telegram
source_kortitleurl posted_attagreturnc C st t |pd}d| ddg}|r|dt | |r$|d| |d| |d| d d |S )
N u 🏛️ u
인사동정u 분류: u 게시일: u 제목: z 🔗 상세 보기
)_escapeappendjoin)r r r r r title_sbody r 3/var/www/html/govbot/app/workers/full_crawl_once.py_msg s
r max_pages sleep_sec
send_telegramc C s t dd}|r t ng }d}td| d D ]q}tt||d}|s-td| d |S d}|D ]@} | d }
t|
r
r> c
C s\ t dd}|r t ng }d}td| d D ]}tt||d}|s-td| d |S d}|D ]d} | d }
t|
r sH 4
rH c C s t jdd} | jdtddd | jdtddd | jd td
d d | jd d
dd | }t|j|j|j }t
|j |j|j }t d| d| d S )Nz0One-off full crawl (all pages) for MOTIE & MOEF.) descriptionz--motie-max-pages zMax pages to try for MOTIE)typedefaulthelpz--moef-max-pageszMax pages to try for MOEFz--sleepgffffff?zSleep seconds between pagesz--send-telegram
store_truez&Also broadcast telegram (default: off))actionrM z[SUMMARY] MOTIE new: z , MOEF new: )
argparseArgumentParser add_argumentintfloat
parse_argsr> motie_max_pagesr5 r rH moef_max_pagesr1 )parserargs motie_newmoef_newr r r maina s r\ __main__)NN)#r- r4 rP pathlibr dotenvr htmlr r __file__resolveparentsBASE_DIRapp.crawlers.motier app.crawlers.moefr app.services.supabase_servicer r r
r r app.services.notify_servicer
strr rS rT boolr> rH r\ __name__r r r r s4
#