* improved logic that prevents overlapping runs, added logic to kill the process if it is stuck or doesn't exit properly
* Improve lock file logic
Improve lock file logic to release lock if the script didn't finish cleanly on a prior run. Otherwise shouldn't interrupt actual run.
* Tidy up readme for docker
* Optimizations of dockerfile
Massive test optimizations with drop in image size to about 256mb from about 1.2 gb. Drawback is that I currently have to keep the dockerfile playwright version matched to the package.json version
* further optimizations
Removed redundant (hopefully) sessions directory creation during build
* Fix docker cron dependencies
Small fix that should make cron run properly
* Major docker update!
- **Dockerfile rewritten as a multi-stage build**
- Split into a “builder” stage (`node:18-slim`) to install dependencies and compile TypeScript, and a “runtime” stage (official Playwright image) to run the script.
- This keeps build tools and dependencies out of the final image, making it smaller, faster to pull, and more secure.
- **Entrypoint script (`entrypoint.sh`)**
- Introduced an entrypoint that runs inside the container at startup to:
1. Set the container’s timezone (`TZ`) correctly, based on the environment or defaulting to UTC.
2. Validate that the user provided a `CRON_SCHEDULE` (exiting early with an error if missing).
3. Optionally perform an initial run of the script immediately (when `RUN_ON_START=true`), without any random sleep.
- Centralizing setup in an entrypoint keeps the Dockerfile simpler and ensures proper signal handling.
- **`run_daily.sh` improvements**
- Removed custom browser-path override so Playwright uses bundled browsers in the official image.
- Added a lock using `flock` to prevent overlapping runs if a previous run is still in progress.
- Retained the random sleep between 5 and 50 minutes before each run.
- Logs are timestamped and clearly report success or failure.
- **Cron template tweaks**
- Updated `src/crontab.template` so that each job line redirects both stdout and stderr into Docker’s stdout (`>> /proc/1/fd/1 2>&1`), making it easy to view logs via `docker logs`.
- **Initial-run logic**
- The entrypoint checks `RUN_ON_START=true` and, if set, invokes `npm start` immediately (without random sleep). This provides an immediate first execution on container startup.
- Scheduled runs via cron still go through the normal `run_daily.sh` (with sleep and locking).
- **Cron logging and visibility**
- By redirecting cron job output to the container’s stdout, all logs (initial run and scheduled runs) appear in `docker logs`, avoiding the need to tail log files manually.
- **Error handling and validation**
- Entry point exits early if `CRON_SCHEDULE` is missing, preventing silent misconfiguration.
- If the initial run fails, it logs a warning but still starts cron so future scheduled runs can proceed.
- `run_daily.sh` will exit early if a previous run is still active (locking), avoiding overlapping executions.
* Docker (multi-stage) improvements
- added cron logging in entrypoint and fixed timezone support for cron-invoked script runs
- further optimized multi-stage dockerfile
- bumped playwright version to 1.52.0 in dockerfile and package.json
- added customization and enable/disable randomization for cron start times
- optionally add container health monitor and resource limits in compose.yaml
* 1.4.12
* Update README.md
* Update package.json
* Update package.json
* 1.5 initial
- Added parallel mode (experimental, likely no Docker supported)
- Added chalk for clearer logging
- Added support for "SearchOnBing" Activities
- Added more configurable options for certain things
- Redone some of the popup/banner clicking for searching (Redo the entire "popup" clicking, so they're more specifically targeted)
- Axios proxy is now optional in the config
- Fingerprint saving is now optional for desktop and mobile
There needs to be many changes for Docker support, including parallel, the new config settings and general testing!
This is still highly experimental, if you use Docker or want something more stable, use the version before this commit!
* Add queries.json to build
* fix(Login): update URL within authorization loop to reflect current page (#210)
* Many changes
- Updated Packages
- Fixed mobile searches erroring out for dashboard data
- Reworked "bad page" detection
- Catching more errors
- Reworked the search and "close tabs"
- More fixes to the login
- Fixed to paralell and clustering, thanks to @AariaX
* Docker 1.5 preliminary support (#211)
* Basic docker functionality for 1.5
Preliminary docker support for 1.5. Requires headless=true, clusters=1
* Tidy up timezone, add TZ to compose file
Minor changes that should improve timezone handling, and (hopefully) improve scheduling function
* updated readme to simplify and clarify docker instructions
also removed env vars from table
* Fix syntax for cron
* Fix scheduling, add .gitattributes to normalize line endings
fixed line endings caused by Windows in crontab.template and run_daily.sh, which were breaking cron and script execution in the Docker container.
* Removed unnecessary scheduling key from config.json
This key isn't necessary for docker or the base script.
* Basic docker functionality for 1.5
Preliminary docker support for 1.5. Requires headless=true, clusters=1
Tidied up timezone, add TZ to compose file
Minor changes that should improve timezone handling, and (hopefully) improve scheduling function
updated readme to simplify and clarify docker instructions
also removed env vars from table
Fixed syntax for cron
Fixed scheduling, add .gitattributes to normalize line endings
Fixed line endings caused by Windows in crontab.template and run_daily.sh, which were breaking cron and script execution in the Docker container.
Removed unnecessary scheduling key from config.json
This key isn't necessary for docker or the base script.
* Improve scheduling handling, show logs in console
Fixes scheduling when RUN_ON_START=true, and fixes scheduled runs not appearing in docker logs.
* Update compose.yaml
revert service and container name, revert volumes for better generalization, add tips to environment to set scheduling, timezone and whether to run on container start
* Update README.md
proper container name
Co-authored-by: AariaX <196196201+AariaX@users.noreply.github.com>
---------
Co-authored-by: AariaX <196196201+AariaX@users.noreply.github.com>
* Fixes
- Reworked some of the point counting
- Reverted back to the "playwright" package
- Fixed error throw for emailPrefill
* Update config.json
* Add pre-build script
* Update package.json
* Handle 2FA in parallel mode (#219)
* catch error in reloadBadPage (#220)
* Use pre-build and simplify dockerfile (#218)
This uses the new pre-build script included in package.json to handle deps greatly simplifying the dockerfile.
* Small improvements
* Small fixes
- Fixed log spam for "Waiting for authorization"
- Increased wait from 2 to 5 seconds
- Increased search to "safer" values for default
* Experimenting with selectors
Seeing #223 I want to try if this is a good new addition, since for most user this SHOULD work just as good as clicking the entire box.
* More stuff
- Added ability to exclude logs by their function name
- Now caching config settings
* fix: don't retry on 0 (#228)
* Improvements
- Check if searches for mobile are enabled before creating the new page in the browser
- Return message if mobile search data cannot be found
- Added more selectors for coupons
* Improve Popup Dismissal
- Now executes in Parallel
- Respects a timeout of 1 second
---------
Co-authored-by: AariaX <196196201+AariaX@users.noreply.github.com>
Co-authored-by: mgrimace <55518507+mgrimace@users.noreply.github.com>
- Update run_daily.sh to use compose.yaml env var overrides if present on scheduled jobs, not just first run. Defaults back to config.json values if no override specified.
- update sample compose.yaml with volume mapping to keep config.json, accounts.json, and sessions folder persistent on local machine (e.g., to keep data persistent through updates).
* Improve env var handling, clarify instructions
updateConfig.js will update dist/config.json with any values specified in the docker compose file as environmental variables (env vars). If not specified it will use the default values in src/config.json (the 'usual' place where folks can customize their config).
A user can make changes to an env var (e.g., disabling Scroll Random Results), then docker compose up -d to quickly restart the container with the change.
* minor update to env vars in table
Make sure to change your compose so the updated flattened values work.
* TZ handling for cron runs of the script
docker logs netsky should now show the proper time zone for script runs that were initiated via cron schedule.
* Script will run on container start, and continue to run daily at 5:00 am.
The goal of this update was to add in-built cronjob support so that the script runs immediately on container start, then will schedule the script to run the daily at 5:00 am without requiring the user to manually create a cronjob. After starting the container and completing the first run, the container should remain active (running) but idle until the next scheduled run. Ideally, the next step will be to include support for environmental variables that will allow the user to customize when the script scheduled runs occur within the docker compose.yaml.
* added --build to docker compose command instructions
building/rebuilding the image (and the script) may reduce problems if folks make changes to their configs
* Updated the crontab default to run twice daily by default, 5am and 11am
* Update crontab
Newline needed
* Environmental variable support, custom scheduling
This is a massive update that includes docker environmental variable support for all config options. If specified, the custom env var value will supersede the default values in the config.json. You do not need to change the values in the config.json if you set a custom env var. For example, if you have headless=false in the config.json, but set it as true in the compose.yaml, it will run as HEADLESS=true. If you do not specify a particular env var, the script will automatically use the default value. Please test and contribute wherever possible!
* Fix scheduled runs, tidied up readme
* timezone support
Minor adjustment to attempt to fix timezone env var support
* Run on start, npm script env
Hopefully two fixes here, the container should again, by default, run on start, then continue per schedule. This defaults to true; however, can be optionally over-ridden in the compose.yaml with the env var RUN_ON_START=false. If set to false, the script should only run on the specified cron schedule. I also passed the TZ write into the npm start command, rather than just the container overall so it should show the proper time.
* Keep docker running after run_on_start to wait for cron
Accidentally set it up like a run_once, this should keep the container active after run_on_start to wait for the next schedule.
* improve order of cmd
This should (1) set time zone for script, (2) run once if set, then (3) idle for scheduled cron job. It was a little hacky before, this should be tidier.
* updated compose to auto-restart container
Noticed the container was exited this am, likely after automated updated to system, adding restart: unless-stopped to compose should keep container running in such circumstances.
* Update env vars in readme
