Commit Graph

78 Commits

Author SHA1 Message Date
32927aea3d Formatting pass 2021-01-18 08:37:32 -06:00
fe7b93d6d3 Added max time remaining debug logging to cron upload 2021-01-16 14:35:59 -06:00
6282881c4a Changed cron uploader to better handle concurrent run attmpts 2021-01-16 14:35:54 -06:00
e7d11f2310 Converted cron_upload to be able to run per-user, using user API keys 2021-01-16 14:35:04 -06:00
72268d529d Moved user options to separate page, modularized the settings page code so that it can be re-used to simplify and standardize user settings in the same manner as global settings 2021-01-16 14:26:17 -06:00
ba982e4451 Image to Post - cron upload 2020-10-26 10:53:28 -05:00
45d38d8833 Added more specific upload error message that includes detected mimetype 2020-10-26 07:27:16 -05:00
984c9702ec The great MIMEing 2020-06-25 00:42:34 +01:00
919a3039c4 Added user API key system 2020-06-23 15:14:04 +01:00
30f62c2ff8 Fixed transaction issue between cron uploader and bulk import 2020-06-23 15:11:35 +01:00
9b9f1d0341 Cleaned up some warnings in bulk import/export extension
Added transactions to bulk import
Renamed beginTransaction to begin_transaction for naming consistency
Updated cron uploader to handle bulk import transactions
2020-06-23 15:07:00 +01:00
f6923af8ab Added source logging to cron upload output when all sources are enabled 2020-06-22 18:03:03 +01:00
86f7a06ed0 New options for cron uploader:
Logging level
Including all logs in output
Stop on error instead of continuing
2020-06-03 20:00:54 +01:00
e1aefb78ab Fix for cron uploader issue with root paths ending in a slash 2020-06-03 19:59:29 +01:00
12c331cbd2 Removed extra tag set from cron uploader, no longer needed now that it runs as a user 2020-06-03 19:58:57 +01:00
bccb206369 formatting pass 2020-06-03 19:58:13 +01:00
007e07e507 Various changes to cron uploader:
Removed count limit, the cron job now checks the max PH execution time and auto-stops itself at 80% of that value.

Now skips os-specific image cache files like thumbs.db and the __macosx folder.

Changed failed folder re-deployment to allow re-deploying to populated queue, making it easier to re-process lots of failed batches all at once.

Changed page to output as a stream, allowing a long-running process to provide output as it runs rather than just at the very end.

Changed import loop to use the yield convention, allowing faster consumption of found files and lower memory use overall.
2020-06-03 19:57:27 +01:00
e46b319295 set theme classes 2020-02-04 00:46:36 +00:00
9eb5acf2dc use strict types 2020-01-26 16:27:56 +00:00
70db0ce5bd flash_message -> page->flash, with no cookies 2019-12-15 19:47:18 +00:00
247cfcbd77 Avoid shadowing global variable
When we aren't referencing the current `global $user`, we should give it
a different name to avoid confusion
2019-11-11 16:43:42 +00:00
c7d90c2df7 nits 2019-11-04 01:04:12 +00:00
d17e207984 Have get_arg never return null
90% of places assume it will never return null, and they will break in
weird ways if it does return null
2019-11-04 00:40:10 +00:00
55c6854003 formatting 2019-11-02 19:57:34 +00:00
d605e0e572 Added cron_admin permission 2019-10-17 14:26:14 -05:00
40269a6f4a Cron uploader enhancements and bug fixes 2019-10-10 10:16:15 -05:00
30698fefdc remove a bunch of dead variables and things 2019-10-02 11:23:57 +01:00
8f688fd2c6 lints 2019-10-02 10:10:47 +01:00
a7bddb1dac get rid of is_admin, fixes #676 2019-09-29 19:01:09 +01:00
de98e86938 ExtensionInfo conversions what have I done 2019-08-07 15:40:01 -05:00
c24a6e9b97 formatting pass 2019-06-21 09:12:44 +01:00
921ec9a7bb Adjusted cron upload for new merged flag, and to make sure tags merge properly 2019-06-19 20:20:52 -05:00
5eb4a66ab7 Added merged indicator to DataUploadEvent and ImageAddEvent
Changed merge process so that the ID of the merged image can make it back through the event chanin
2019-06-19 20:19:38 -05:00
5a30ce1c83 Reverted removal of latter tag write 2019-06-19 18:59:51 -05:00
826c623538 PageMode constants 2019-06-18 21:04:31 -05:00
1fe18e7573 Missed a dir name 2019-06-15 12:52:22 -05:00
8b531c04a2 removed SQLERROR escape from cron uploader, not necessary now that it is individualizing transactions.
Change cron uploader to use constants for dir and config names
2019-06-15 12:51:04 -05:00
ab9389007f Changed key-generation process for cron upload so it doesn't endlessly generate new keys before the user first hits the same buttons in settings. 2019-06-15 11:51:03 -05:00
0202597f88 Added lock file usage to cron uploader to prevent concurrent runs.
Changed extension manager to allow author to be a comma-separated list.
2019-06-15 11:19:51 -05:00
064b24ffc1 formatting pass 2019-06-14 13:47:50 +01:00
d1102cd635 Merge from sanmadjack:develop 2019-06-14 13:46:55 +01:00
f078b283bd pull a bunch of small fixes from #659 2019-06-14 13:16:58 +01:00
68c3e5ea42 Changed cron upload so that an unrecognised file type results in an error instead of a success 2019-06-13 18:17:16 +01:00
cb24ac69ab Changes to cron upload:
Added transaction handling so that subsequent errors don't result in images that have already moved to the uploaded folder from being wiped from the database.
Changed output folders to use subfolders based on the timestamp of the current run. This is to prevent writing over files in the error folder that happen to have the same name and path, effectively losing the data.
Added additional error and information logging, and a final count of imported/merged/failed.
2019-06-12 17:50:00 -05:00
97abeb5254 Added option to detect file type based on header bytes 2019-06-12 17:35:11 -05:00
4410baeb9c Changed cron upload so that an unrecognised file type results in an error instead of a success 2019-06-09 14:17:13 -05:00
aef455949b Added escape to cron upload to stop the process when a transaction-breaking error occurs. 2019-06-02 13:38:25 -05:00
38badf7e45 Changed cron import to output imported/failed files to subdirectories matching the imported file's original subdirectory 2019-06-02 13:34:24 -05:00
5a2f893667 Changed cron upload new image tagging to work with tag event's requirement for tags to not be empty. 2019-06-01 12:17:38 -05:00
3e2a0ea3b5 Brought cron upload tag handling inline with everything else 2019-06-01 12:12:36 -05:00