RepoMirrors
/
hydrus
mirror of https://github.com/hydrusnetwork/hydrus.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Version 490 

closes #1173
This commit is contained in:
Hydrus Network Developer 2022-06-29 15:52:53 -05:00
parent 9df036c0ec
commit 3a143af7a5
22 changed files with 515 additions and 572 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

68
docs/changelog.md
View File

@ -3,6 +3,31 @@
!!! note
This is the new changelog, only the most recent builds. For all versions, see the [old changelog](old_changelog.html).
## [Version 490](https://github.com/hydrusnetwork/hydrus/releases/tag/v490)
### misc
* fixed a stupid bug that meant the image caches were initialising with default values (as under _speed and memory_) until you opened and OKed the options dialog (or did some other options-refresh events). sorry for the trouble, please enjoy some smoother image browsing.
* mr bones now shows more numbers, and in a neater table. it should be clearer what the percentages are for now, too
* the _manage->regenerate_ thumbnail menu has additional quick maintenance commands for presence and integrity checks and regenerating data in the similar files system
* wrote a new 'special duplicate' button for the edit shortcut set dialog. the list on this dialog doesn't allow duplicates (which meant the old 'duplicate' button was doing nothing), so this duplicates the current actions with 'incremented' shortcut keys. 'a' becomes 'b', 'ctrl+5' becomes 'ctrl+6', and so on. it doesn't always work, but if you want to make ten shortcuts for setting rating 1-10, this should help
* fixed an issue where the thumbnail banner text and the media viewer background text was not changing size or font according to QSS stylesheet rules (issue #1173)
* SIGTERM should now cause a clean program exit (previously it killed the GUI App but left some daemon threads alive for thirty seconds or more). unlike SIGINT, it will not ask you if you are sure you want to exit or if you would like to do shutdown maintenance--it just closes the client promptly
* fixed a bug in last week's importer page status improvements--the hard drive import page wasn't showing all the updates it should have
* brushed up some backup help
### file services
* fixed a bug where advanced users could set 'all known files'/'all known tags' on a search dropdown. this search domain is not supported
* in the archive/delete filter, if the current location is 'all my files' and the files being deleted are only in one local file domain, the surplus 'all my files' will no longer appear at the top of the filter's commit dialog
* the file services in the thumbnail select/remove menu are now sorted in the same order as the file domain button in search dropdowns
* the thumbnail select/remove menus now exclude 'all my files' and 'all local files' if those choices are redundant (e.g. if you only have files in 'my files', 'all my files' will be hidden)
* fixed some incorrect 'delete from x' actions appearing in thumbnail right-click menus
### orphan files
* there's a persistent processing bug some users have where some update files are missing but they won't redownload correctly. I think I fix that this week naturally so existing maintenance routines will now be able to fix it themselves after another round
* fixed some issues related to deleting files from the repository updates file domain.
* the 'clear orphan file records' maintenance command now fixes the 'all my files' umbrella services as well as the 'all local files' one. it also has nicer description, does some additional file-removal cleanup, and triggers a file recount if problems are found
* moved 'clear orphan files' to the 'files' maintenance menu
## [Version 489](https://github.com/hydrusnetwork/hydrus/releases/tag/v489)
### downloader pages
@ -297,46 +322,3 @@
* if you have a really really huge session, the client is now more careful about not booting delayed background tasks like subscriptions until the session is in place
* on 'migrate database', the thumbnail size estimate now has a min-max range and a tooltip to clarify that it is an estimate
* fixed a bug in the new 'sort by file hash' pre-sort when applying system:limit
## [Version 479](https://github.com/hydrusnetwork/hydrus/releases/tag/v479)
### misc
* when shift-selecting some thumbnails, you can now reverse the direction of the select and what you just selected will be deselected, basically a full undo (issue #1105)
* when ctrl-selecting thumbnails, if you add to the selection, the file you click is now focused and always previewed (previously this only happened if there was no focused file already). this is related to the shift-select logic above, but it may be annoying when making a big ctrl-selection of videos etc.. so let me know and I can make this more clever if needed
* added file sort 'file->hash', which sorts pseudorandomly but repeatably. it sounds not super clever, but it will be useful for certain comparison operations across clients
* when you hit 'copy->hash' on a file right-click, it now shows the sha256 hash for quick review
* in the duplicate filter, the zoom locking tech now works better™ when one of the pair is portrait and the other landscape. it now tries to select either width or height to lock both when going AB and BA. it also chooses the 'better' of width or height by choosing the zoom that'll change the size less radically. previously, it could do width on AB and height on BA, which lead to a variety of odd situations. there are probably still some issues here, most likely when one of the files almost exactly fills the whole canvas, so let me know how you get on
* webps with transparency should now load correct! previously they were going crazy in the transparent area. all webps are scheduled a thumbnail regen this week
* when import folders run, the count on their progress bar now ignores previous failed and ignored entries. it should always start 0, like 0/100, rather than 20/120 etc...
* when import folders run, any imports where the status type is set to 'leave the file alone' is now still scanned at the end of a job. if the path does not exist any more, it is removed from the import list
* fixed a typo bug in the recent delete code cleanup that meant 'delete files after export' after a manual export was only working on the last file in the selection. sorry for the trouble!
* the delete files dialog now starts with keyboard focus on the action radiobox (it was defaulting to ok button since I added the recent panel disable tech)
* if a network job has a connection error or serverside bandwidth block and then waits before retrying, it now checks if all network jobs have just been paused and will not reattempt the connection if so (issue #1095)
* fixed a bug in thumbnail fallback rendering
* fixed another problem with cloudscraper's new method names. it should work for users still on an old version
* wrote a little 'extract version' sql and bat file for the db folder that simply pull the version from the client.db file in the same directory. I removed the extract options/subscriptions sql scripts since they are super old and out of date, but this general system may return in future
### file history chart
* added 'archive' line to the file history chart. this isn't exactly (current_count - inbox_count), but it pretty much is
* added a 'show deleted' checkbox to the file history chart. it will recalculate the y axis range on click, so if you have loads of deleted files, you can now hide them to see current better
* improved the way data is aggregated in the file history chart. diagonal lines should be reduced during any periods of client import-inactivity, and spikes should show better
* also bumped the number of steps up to 8,000, so it should look nice maximised on a 4k
* the file history chart now remembers its last size and position--it has an entry under options->gui
### client api
* thanks to a user, the Client API now accepts any file_id, file_ids, hash, or hashes as arguments in any place where you need to specify a file or files
* like 'return_hashes', the 'search_files' command in the Client API now takes an optional 'return_file_ids' parameter, default true, to turn off the file ids if you only want hashes
* added 'only_return_basic_information' parameter, default false, to 'get_metadata' call, which is fast for first-time requests (it is slim but not well cached) and just delivers the basics like resolution and file size
* added unit tests and updated the help to reflect the above
* client api version is now 29
### help
* split up the 'more files' help section into 'powerful searching' and 'exporting files', both still under the 'next steps' section
* moved the semi-advanced 'OR' section from 'tags' to 'searching'
* brushed up misc help
* a couple of users added some misc help updates too, thank you!
### misc boring cleanup
* cleaned up an old wx label patch
* cleaned up an old wx system colour patch
* cleaned up some misc initialisation code

31
docs/getting_started_installing.md
View File

@ -160,18 +160,23 @@ _All that said, and while updating is complex and every client is different, one
Hydrus's database engine, SQLite, is excellent at keeping data safe, but it cannot work in a faulty environment. Ways in which users of hydrus have damaged/lost their database:
* Hard drive hardware failure (age, bad ventilation, bad cables, etc...)
* Lightning strike on non-protected socket or rough power cut on non-UPS'd power supply
* RAM failure
* Motherboard/PSU power problems
* Accidental deletion
* Accidental overwrite (usually during a borked update)
* Encrypted partition auto-dismount/other borked settings
* Cloud backup interfering with ongoing writes
* Network drive location not guaranteeing accurate file locks
* Hard drive hardware failure (age, bad ventilation, bad cables, etc...)
* Lightning strike on non-protected socket or rough power cut on non-UPS'd power supply
* RAM failure
* Motherboard/PSU power problems
* Accidental deletion
* Accidental overwrite (usually during a borked update)
* Encrypted partition auto-dismount/other borked settings
* Cloud backup interfering with ongoing writes
* A laptop that incorrectly and roughly disconnected an external USB drive on every sleep
* Network drive location not guaranteeing accurate file locks
* Windows NVMe driver bugs necessitating a different SQLite journalling method
Some of those you can mitigate (don't run the database over a network!) and some will always be a problem, but if you have a backup, none of them can kill you.
!!! note "This mostly means your database, not your files"
Note that nearly all the serious and difficult-to-fix problems occur to the _database_, which is four large .db files, not your media. All your images and movies are read-only in hydrus, and there's less worry if they are on a network share with bad locks or a machine that suddenly loses power. The database, however, maintains a live connection, with regular complex writes, and here a hardware failure can lead to corruption (Basically the failure scrambles the data that is written, so when you try to boot back up, a small section of the database is incomprehensible garbage).
If you do not already have a backup routine for your files, this is a great time to start. I now run a backup every week of all my data so that if my computer blows up or anything else awful happens, I'll at worst have lost a few days' work. Before I did this, I once lost an entire drive with tens of thousands of files, and it felt awful. If you are new to saving a lot of media, I hope you can avoid what I felt. ;\_;
I use [ToDoList](http://abstractspoon.com/) to remind me of my jobs for the day, including backup tasks, and [FreeFileSync](http://sourceforge.net/projects/freefilesync/) to actually mirror over to an external usb drive. I recommend both highly (and for ToDoList, I recommend hiding the complicated columns, stripping it down to a simple interface). It isn't a huge expense to get a couple-TB usb drive either--it is **absolutely** worth it for the peace of mind.
@ -184,7 +189,7 @@ By default, hydrus stores all your user data in one location, so backing up is s
Once you have your location set up, you can thereafter hit _database->update database backup_. It will lock everything and mirror your files, showing its progress in a popup message. The first time you make this backup, it may take a little while (as it will have to fully copy your database and all its files), but after that, it will only have to copy new or altered files and should only ever take a couple of minutes.
Advanced users who have migrated their database across multiple locations will not have this option--use an external program in this case.
Advanced users who have migrated their database and files across multiple locations will not have this option--use an external program in this case.
#### the powerful (and best) way - using an external program
:
@ -200,7 +205,7 @@ By default, hydrus stores all your user data in one location, so backing up is s
Note it has 'file time and size' and 'mirror' as the main settings. This quickly ensures that changes to the left-hand side are copied to the right-hand side, adding new files and removing since-deleted files and overwriting modified files. You can save a backup profile like that and it should only take a few minutes every week to stay safely backed up, even if you have hundreds of thousands of files.
Shut the client down while you run the backup, obviously.
**Shut the client down while you run the backup, obviously.**
##### A few options
@ -223,12 +228,12 @@ Almost every OS you can name.
There is significantly more information about the database structure [here](database_migration.md).
I recommend you always backup before you update, just in case there is a problem with my code that breaks your database. If that happens, please [contact me](contact.md), describing the problem, and revert to the functioning older version. I'll get on any problems like that immediately.
I recommend you always backup before you update, just in case there is a problem with my update code that breaks your database. If that happens, please [contact me](contact.md), describing the problem, and revert to the functioning older version. I'll get on any problems like that immediately.
## backing up with not much space { id="backing_up_small" }
If you decide not to maintain a backup because you cannot afford drive space for all your files, please please at least back up your actual database files. Use FreeFileSync or a similar program to back up the four 'client*.db' files in install_dir/db when the client is not running. Just make sure you have a copy of those files, and then if your main install becomes damaged, we will have a reference to either roll back to or manually restore data from. Even if you lose a bunch of media files in this case, with an intact database we'll be able to schedule recovery of anything with a URL.
If you are really short on space, note also that the database files are very compressible. A very large database where the four files add up to 70GB can compress down to 17GB zip with 7zip on default settings. This obviously takes some additional time to do, but if you are really short on space it may be the only way it fits, and if your only backup drive is a slow USB stick, then you might actually save time from not having to transfer the other 53GB! Media files (jpegs, webms, etc...) are generally not very compressible, usually 5% at best, so it is usually not worth trying.
If you are really short on space, note also that the database files are very compressible. A very large database where the four files add up to 70GB can compress down to 17GB zip with 7zip on default settings. Better compression ratios are possible if you make sure to put all four files in the same archive and turn up the quality. This obviously takes some additional time to do, but if you are really short on space it may be the only way it fits, and if your only backup drive is a slow USB stick, then you might actually save time from not having to transfer the other 53GB! Media files (jpegs, webms, etc...) are generally not very compressible, usually 5% at best, so it is usually not worth trying.
It is best to have all four database files. It is generally easy and quick to fix problems if you have a backup of all four. If client.caches.db is missing, you can recover but it might take ten or more hours of CPU work to regenerate. If client.mappings.db is missing, you might be able to recover tags for your local files from a mirror in an intact client.caches.db. However, client.master.db and client.db are the most important. If you lose either of those, or they become too damaged to read and you have no backup, then your database is essentially dead and likely every single archive and view and tag and note and url record you made is lost. This has happened before, do not let it be you.

4
docs/launch_arguments.md
View File

@ -38,14 +38,14 @@ But you can also check it in _help->about_. A handful of database operations (PT
##**`--db_journal_mode {WAL,TRUNCATE,PERSIST,MEMORY}`**
Change the _journal_ mode of the SQLite database. The default is WAL, which works great for SSD drives, but if you have a very old or slow drive, a different mode _may_ work better. Full docs are [here](https://sqlite.org/pragma.html#pragma_journal_mode).
Change the _journal_ mode of the SQLite database. The default is WAL, which works great for almost all SSD drives, but if you have a very old or slow drive, or if you encounter 'disk I/O error' errors on Windows with an NVMe drive, try TRUNCATE. Full docs are [here](https://sqlite.org/pragma.html#pragma_journal_mode).
Briefly:
* WAL - Clever write flushing that takes advantage of new drive synchronisation tools to maintain integrity and reduce total writes.
* TRUNCATE - Compatibility mode. Use this if your drive cannot launch WAL.
* PERSIST - This is newly added to hydrus. The ideal is that if you have a high latency HDD drive and want sync with the PTR, this will work more efficiently than WAL journals, which will be regularly wiped and recreated and be fraggy. Unfortunately, with hydrus's multiple database file system, SQLite ultimately treats this as DELETE, which in our situation is basically the same as TRUNCATE, so does not increase performance. Hopefully this will change in future.
* MEMORY - Danger mode. Extremely fast, but you had better guarantee a lot of free ram.
* MEMORY - Danger mode. Extremely fast, but you had better guarantee a lot of free ram and no unclean exits.
##**`--db_transaction_commit_period DB_TRANSACTION_COMMIT_PERIOD`**

25
docs/old_changelog.html
View File

@ -33,6 +33,31 @@
<div class="content">
<h3 id="changelog"><a href="#changelog">changelog</a></h3>
<ul>
<li><h3 id="version_490"><a href="#version_490">version 490</a></h3></li>
<ul>
<li>misc:</li>
<li>fixed a stupid bug that meant the image caches were initialising with default values (as under _speed and memory_) until you opened and OKed the options dialog (or did some other options-refresh events). sorry for the trouble, please enjoy some smoother image browsing.</li>
<li>mr bones now shows more numbers, and in a neater table. it should be clearer what the percentages are for now, too</li>
<li>the _manage->regenerate_ thumbnail menu has additional quick maintenance commands for presence and integrity checks and regenerating data in the similar files system</li>
<li>wrote a new 'special duplicate' button for the edit shortcut set dialog. the list on this dialog doesn't allow duplicates (which meant the old 'duplicate' button was doing nothing), so this duplicates the current actions with 'incremented' shortcut keys. 'a' becomes 'b', 'ctrl+5' becomes 'ctrl+6', and so on. it doesn't always work, but if you want to make ten shortcuts for setting rating 1-10, this should help</li>
<li>fixed an issue where the thumbnail banner text and the media viewer background text was not changing size or font according to QSS stylesheet rules (issue #1173)</li>
<li>SIGTERM should now cause a clean program exit (previously it killed the GUI App but left some daemon threads alive for thirty seconds or more). unlike SIGINT, it will not ask you if you are sure you want to exit or if you would like to do shutdown maintenance--it just closes the client promptly</li>
<li>fixed a bug in last week's importer page status improvements--the hard drive import page wasn't showing all the updates it should have</li>
<li>brushed up some backup help</li>
<li>.</li>
<li>file services:</li>
<li>fixed a bug where advanced users could set 'all known files'/'all known tags' on a search dropdown. this search domain is not supported</li>
<li>in the archive/delete filter, if the current location is 'all my files' and the files being deleted are only in one local file domain, the surplus 'all my files' will no longer appear at the top of the filter's commit dialog</li>
<li>the file services in the thumbnail select/remove menu are now sorted in the same order as the file domain button in search dropdowns</li>
<li>the thumbnail select/remove menus now exclude 'all my files' and 'all local files' if those choices are redundant (e.g. if you only have files in 'my files', 'all my files' will be hidden)</li>
<li>fixed some incorrect 'delete from x' actions appearing in thumbnail right-click menus</li>
<li>.</li>
<li>orphan files:</li>
<li>there's a persistent processing bug some users have where some update files are missing but they won't redownload correctly. I think I fix that this week naturally so existing maintenance routines will now be able to fix it themselves after another round</li>
<li>fixed some issues related to deleting files from the repository updates file domain.</li>
<li>the 'clear orphan file records' maintenance command now fixes the 'all my files' umbrella services as well as the 'all local files' one. it also has nicer description, does some additional file-removal cleanup, and triggers a file recount if problems are found</li>
<li>moved 'clear orphan files' to the 'files' maintenance menu</li>
</ul>
<li><h3 id="version_489"><a href="#version_489">version 489</a></h3></li>
<ul>
<li>downloader pages:</li>

32
hydrus/client/ClientController.py
View File

@ -547,7 +547,7 @@ class Controller( HydrusController.HydrusController ):
elif sig == signal.SIGTERM:
QP.CallAfter( QW.QApplication.instance().quit )
self.Exit()
@ -1004,20 +1004,6 @@ class Controller( HydrusController.HydrusController ):
HydrusController.HydrusController.InitModel( self )
self.frame_splash_status.SetText( 'initialising managers' )
self.frame_splash_status.SetSubtext( 'image caches' )
# careful: outside of qt since they don't need qt for init, seems ok _for now_
self._caches[ 'images' ] = ClientCaches.ImageRendererCache( self )
self._caches[ 'image_tiles' ] = ClientCaches.ImageTileCache( self )
self._caches[ 'thumbnail' ] = ClientCaches.ThumbnailCache( self )
self.bitmap_manager = ClientManagers.BitmapManager( self )
self.frame_splash_status.SetSubtext( 'services' )
self.services_manager = ClientServices.ServicesManager( self )
self.frame_splash_status.SetSubtext( 'options' )
self.options = self.Read( 'options' )
@ -1033,6 +1019,22 @@ class Controller( HydrusController.HydrusController ):
self.frame_splash_status.SetSubtext( 'image caches' )
self._caches[ 'images' ] = ClientCaches.ImageRendererCache( self )
self._caches[ 'image_tiles' ] = ClientCaches.ImageTileCache( self )
self._caches[ 'thumbnail' ] = ClientCaches.ThumbnailCache( self )
self.frame_splash_status.SetText( 'initialising managers' )
# careful: outside of qt since they don't need qt for init, seems ok _for now_
self.bitmap_manager = ClientManagers.BitmapManager( self )
self.frame_splash_status.SetSubtext( 'services' )
self.services_manager = ClientServices.ServicesManager( self )
# important this happens before repair, as repair dialog has a column list lmao
column_list_manager = self.Read( 'serialisable', HydrusSerialisable.SERIALISABLE_TYPE_COLUMN_LIST_MANAGER )

79
hydrus/client/ClientLocation.py
View File

@ -7,6 +7,85 @@ from hydrus.core import HydrusSerialisable
from hydrus.client import ClientConstants as CC
def FilterOutRedundantMetaServices( list_of_service_keys: typing.List[ bytes ] ):
services_manager = HG.client_controller.services_manager
special_local_file_service_keys = { CC.TRASH_SERVICE_KEY, CC.COMBINED_LOCAL_MEDIA_SERVICE_KEY, CC.LOCAL_UPDATE_SERVICE_KEY }
if len( special_local_file_service_keys.intersection( list_of_service_keys ) ) <= 1:
if CC.COMBINED_LOCAL_FILE_SERVICE_KEY in list_of_service_keys:
list_of_service_keys.remove( CC.COMBINED_LOCAL_FILE_SERVICE_KEY )
local_file_service_keys = set( services_manager.GetServiceKeys( ( HC.LOCAL_FILE_DOMAIN, ) ) )
if len( local_file_service_keys.intersection( list_of_service_keys ) ) <= 1:
if CC.COMBINED_LOCAL_MEDIA_SERVICE_KEY in list_of_service_keys:
list_of_service_keys.remove( CC.COMBINED_LOCAL_MEDIA_SERVICE_KEY )
return list_of_service_keys
def GetPossibleFileDomainServicesInOrder( all_known_files_allowed: bool, only_local_file_domains_allowed: bool ):
services_manager = HG.client_controller.services_manager
service_types_in_order = [ HC.LOCAL_FILE_DOMAIN ]
if not only_local_file_domains_allowed:
advanced_mode = HG.client_controller.new_options.GetBoolean( 'advanced_mode' )
if len( services_manager.GetServices( ( HC.LOCAL_FILE_DOMAIN, ) ) ) > 1 or advanced_mode:
service_types_in_order.append( HC.COMBINED_LOCAL_MEDIA )
service_types_in_order.append( HC.LOCAL_FILE_TRASH_DOMAIN )
if advanced_mode:
service_types_in_order.append( HC.LOCAL_FILE_UPDATE_DOMAIN )
if advanced_mode:
service_types_in_order.append( HC.COMBINED_LOCAL_FILE )
service_types_in_order.append( HC.FILE_REPOSITORY )
service_types_in_order.append( HC.IPFS )
if all_known_files_allowed:
service_types_in_order.append( HC.COMBINED_FILE )
services = services_manager.GetServices( service_types_in_order )
return services
def SortFileServiceKeysNicely( list_of_service_keys ):
services_in_nice_order = GetPossibleFileDomainServicesInOrder( False, False )
service_keys_in_nice_order = [ service.GetServiceKey() for service in services_in_nice_order ]
list_of_service_keys = [ service_key for service_key in service_keys_in_nice_order if service_key in list_of_service_keys ]
return list_of_service_keys
def ValidLocalDomainsFilter( service_keys ):
return [ service_key for service_key in service_keys if HG.client_controller.services_manager.ServiceExists( service_key ) and HG.client_controller.services_manager.GetServiceType( service_key ) == HC.LOCAL_FILE_DOMAIN ]

447
hydrus/client/db/ClientDB.py
View File

@ -1122,71 +1122,92 @@ class DB( HydrusDB.HydrusDB ):
self._controller.pub( 'modal_message', job_key )
orphans_found = False
try:
job_key.SetVariable( 'popup_text_1', 'looking for orphans' )
local_file_service_ids = self.modules_services.GetServiceIds( HC.SPECIFIC_LOCAL_FILE_SERVICES )
jobs = [
( ( HC.LOCAL_FILE_DOMAIN, ), self.modules_services.combined_local_media_service_id, 'my files umbrella' ),
( ( HC.LOCAL_FILE_TRASH_DOMAIN, HC.COMBINED_LOCAL_MEDIA, HC.LOCAL_FILE_UPDATE_DOMAIN, ), self.modules_services.combined_local_file_service_id, 'local files umbrella' )
]
local_hash_ids = set()
for local_file_service_id in local_file_service_ids:
for ( umbrella_components_service_types, umbrella_master_service_id, description ) in jobs:
some_hash_ids = self.modules_files_storage.GetCurrentHashIdsList( local_file_service_id )
umbrella_components_service_ids = self.modules_services.GetServiceIds( umbrella_components_service_types )
local_hash_ids.update( some_hash_ids )
umbrella_components_hash_ids = set()
combined_local_hash_ids = set( self.modules_files_storage.GetCurrentHashIdsList( self.modules_services.combined_local_file_service_id ) )
in_local_not_in_combined = local_hash_ids.difference( combined_local_hash_ids )
in_combined_not_in_local = combined_local_hash_ids.difference( local_hash_ids )
if job_key.IsCancelled():
return
job_key.SetVariable( 'popup_text_1', 'deleting orphans' )
if len( in_local_not_in_combined ) > 0:
# these files were deleted from the umbrella service without being cleared from a specific file domain
# they are most likely deleted from disk
# pushing the 'delete combined' call will flush from the local services as well
self._DeleteFiles( self.modules_services.combined_local_file_service_id, in_local_not_in_combined )
for hash_id in in_local_not_in_combined:
for umbrella_components_service_id in umbrella_components_service_ids:
self.modules_similar_files.StopSearchingFile( hash_id )
umbrella_components_hash_ids.update( self.modules_files_storage.GetCurrentHashIdsList( umbrella_components_service_id ) )
HydrusData.ShowText( 'Found and deleted ' + HydrusData.ToHumanInt( len( in_local_not_in_combined ) ) + ' local domain orphan file records.' )
umbrella_master_hash_ids = set( self.modules_files_storage.GetCurrentHashIdsList( umbrella_master_service_id ) )
if job_key.IsCancelled():
in_components_not_in_master = umbrella_components_hash_ids.difference( umbrella_master_hash_ids )
in_master_not_in_components = umbrella_master_hash_ids.difference( umbrella_components_hash_ids )
return
if len( in_combined_not_in_local ) > 0:
# these files were deleted from all specific services but not from the combined service
# I have only ever seen one example of this and am not sure how it happened
# in any case, the same 'delete combined' call will do the job
self._DeleteFiles( self.modules_services.combined_local_file_service_id, in_combined_not_in_local )
for hash_id in in_combined_not_in_local:
if job_key.IsCancelled():
self.modules_similar_files.StopSearchingFile( hash_id )
return
HydrusData.ShowText( 'Found and deleted ' + HydrusData.ToHumanInt( len( in_combined_not_in_local ) ) + ' combined domain orphan file records.' )
job_key.SetVariable( 'popup_text_1', 'deleting orphans' )
if len( in_components_not_in_master ) > 0:
orphans_found = True
# these files were deleted from the umbrella service without being cleared from a specific file domain
# they are most likely deleted from disk
# pushing the master's delete call will flush from the components as well
self._DeleteFiles( umbrella_master_service_id, in_components_not_in_master )
# we spam this stuff since it won't trigger if the files don't exist on master!
self._ArchiveFiles( in_components_not_in_master )
for hash_id in in_components_not_in_master:
self.modules_similar_files.StopSearchingFile( hash_id )
self.modules_files_maintenance_queue.CancelFiles( in_components_not_in_master )
self.modules_hashes_local_cache.DropHashIdsFromCache( in_components_not_in_master )
HydrusData.ShowText( 'Found and deleted {} files that were in components but not the master {}.'.format( HydrusData.ToHumanInt( len( in_components_not_in_master ) ), description ) )
if job_key.IsCancelled():
return
if len( in_master_not_in_components ) > 0:
orphans_found = True
# these files were deleted from all specific services but not from the combined service
# I have only ever seen one example of this and am not sure how it happened
# in any case, the same 'delete combined' call will do the job
self._DeleteFiles( umbrella_master_service_id, in_master_not_in_components )
HydrusData.ShowText( 'Found and deleted {} files that were in the master {} but not it its components.'.format( HydrusData.ToHumanInt( len( in_master_not_in_components ) ), description ) )
if len( in_local_not_in_combined ) == 0 and len( in_combined_not_in_local ) == 0:
if orphans_found:
for service_id in self.modules_services.GetServiceIds( HC.LOCAL_FILE_SERVICES ):
self._Execute( 'DELETE FROM service_info WHERE service_id = ?;', ( service_id, ) )
else:
HydrusData.ShowText( 'No orphan file records found!' )
@ -1506,6 +1527,8 @@ class DB( HydrusDB.HydrusDB ):
local_file_service_ids = self.modules_services.GetServiceIds( ( HC.LOCAL_FILE_DOMAIN, ) )
# we go nuclear on the umbrella services, being very explicit to catch every possible problem
if service_id == self.modules_services.combined_local_file_service_id:
for local_file_service_id in local_file_service_ids:
@ -1513,7 +1536,10 @@ class DB( HydrusDB.HydrusDB ):
self._DeleteFiles( local_file_service_id, hash_ids, only_if_current = True )
self._DeleteFiles( self.modules_services.trash_service_id, hash_ids )
self._DeleteFiles( self.modules_services.combined_local_media_service_id, hash_ids, only_if_current = True )
self._DeleteFiles( self.modules_services.local_update_service_id, hash_ids, only_if_current = True )
self._DeleteFiles( self.modules_services.trash_service_id, hash_ids, only_if_current = True )
if service_id == self.modules_services.combined_local_media_service_id:
@ -1540,18 +1566,20 @@ class DB( HydrusDB.HydrusDB ):
if service_type not in HC.FILE_SERVICES_WITH_NO_DELETE_RECORD:
# make a deletion record
if only_if_current:
deletee_hash_ids = existing_hash_ids
deletion_record_hash_ids = existing_hash_ids
else:
deletee_hash_ids = hash_ids
deletion_record_hash_ids = hash_ids
if len( deletee_hash_ids ) > 0:
if len( deletion_record_hash_ids ) > 0:
insert_rows = [ ( hash_id, existing_hash_ids_to_timestamps[ hash_id ] if hash_id in existing_hash_ids_to_timestamps else None ) for hash_id in deletee_hash_ids ]
insert_rows = [ ( hash_id, existing_hash_ids_to_timestamps[ hash_id ] if hash_id in existing_hash_ids_to_timestamps else None ) for hash_id in deletion_record_hash_ids ]
num_new_deleted_files = self.modules_files_storage.RecordDeleteFiles( service_id, insert_rows )
@ -1559,7 +1587,7 @@ class DB( HydrusDB.HydrusDB ):
if len( existing_hash_ids_to_timestamps ) > 0:
if len( existing_hash_ids ) > 0:
# remove them from the service
@ -5900,6 +5928,8 @@ class DB( HydrusDB.HydrusDB ):
def _GetServiceInfo( self, service_key ):
# TODO: move this to a clever module, and add a 'clear/recalc service info' func so I'm not doing that manually every time
service_id = self.modules_services.GetServiceId( service_key )
service = self.modules_services.GetService( service_id )
@ -10056,317 +10086,6 @@ class DB( HydrusDB.HydrusDB ):
self._controller.frame_splash_status.SetText( 'updating db to v' + str( version + 1 ) )
if version == 429:
try:
tag_service_ids = set( self.modules_services.GetServiceIds( HC.REAL_TAG_SERVICES ) )
file_service_ids = self.modules_services.GetServiceIds( HC.FILE_SERVICES_WITH_SPECIFIC_TAG_LOOKUP_CACHES )
file_service_ids.add( self.modules_services.combined_file_service_id )
for ( file_service_id, tag_service_id ) in itertools.product( file_service_ids, tag_service_ids ):
subtags_searchable_map_table_name = self.modules_tag_search.GetSubtagsSearchableMapTableName( file_service_id, tag_service_id )
self._Execute( 'CREATE TABLE IF NOT EXISTS {} ( subtag_id INTEGER PRIMARY KEY, searchable_subtag_id INTEGER );'.format( subtags_searchable_map_table_name ) )
self._CreateIndex( subtags_searchable_map_table_name, [ 'searchable_subtag_id' ] )
self._RegenerateTagCacheSearchableSubtagMaps()
except Exception as e:
HydrusData.PrintException( e )
raise Exception( 'The v430 subtag searchable map generation routine failed! The error has been printed to the log, please let hydev know!' )
if version == 430:
try:
# due to a bug in over-eager deletion from the tag definition cache, we'll need to resync chained tag ids
tag_service_ids = self.modules_services.GetServiceIds( HC.REAL_TAG_SERVICES )
for tag_service_id in tag_service_ids:
message = 'fixing up some desynchronised tag definitions: {}'.format( tag_service_id )
self._controller.frame_splash_status.SetSubtext( message )
( cache_ideal_tag_siblings_lookup_table_name, cache_actual_tag_siblings_lookup_table_name ) = ClientDBTagSiblings.GenerateTagSiblingsLookupCacheTableNames( tag_service_id )
( cache_ideal_tag_parents_lookup_table_name, cache_actual_tag_parents_lookup_table_name ) = ClientDBTagParents.GenerateTagParentsLookupCacheTableNames( tag_service_id )
tag_ids_in_dispute = set()
tag_ids_in_dispute.update( self._STS( self._Execute( 'SELECT DISTINCT bad_tag_id FROM {};'.format( cache_actual_tag_siblings_lookup_table_name ) ) ) )
tag_ids_in_dispute.update( self._STS( self._Execute( 'SELECT ideal_tag_id FROM {};'.format( cache_actual_tag_siblings_lookup_table_name ) ) ) )
tag_ids_in_dispute.update( self._STS( self._Execute( 'SELECT DISTINCT child_tag_id FROM {};'.format( cache_actual_tag_parents_lookup_table_name ) ) ) )
tag_ids_in_dispute.update( self._STS( self._Execute( 'SELECT DISTINCT ancestor_tag_id FROM {};'.format( cache_actual_tag_parents_lookup_table_name ) ) ) )
if len( tag_ids_in_dispute ) > 0:
self._CacheTagsSyncTags( tag_service_id, tag_ids_in_dispute )
except Exception as e:
HydrusData.PrintException( e )
message = 'Trying to resync some tag definitions failed! Please let hydrus dev know!'
self.pub_initial_message( message )
try:
domain_manager = self.modules_serialisable.GetJSONDump( HydrusSerialisable.SERIALISABLE_TYPE_NETWORK_DOMAIN_MANAGER )
domain_manager.Initialise()
#
domain_manager.OverwriteDefaultParsers( [
'8chan.moe thread api parser',
'e621 file page parser'
] )
#
domain_manager.TryToLinkURLClassesAndParsers()
#
self.modules_serialisable.SetJSONDump( domain_manager )
except Exception as e:
HydrusData.PrintException( e )
message = 'Trying to update some parsers failed! Please let hydrus dev know!'
self.pub_initial_message( message )
if version == 431:
try:
new_options = self.modules_serialisable.GetJSONDump( HydrusSerialisable.SERIALISABLE_TYPE_CLIENT_OPTIONS )
old_options = self._GetOptions()
SORT_BY_LEXICOGRAPHIC_ASC = 8
SORT_BY_LEXICOGRAPHIC_DESC = 9
SORT_BY_INCIDENCE_ASC = 10
SORT_BY_INCIDENCE_DESC = 11
SORT_BY_LEXICOGRAPHIC_NAMESPACE_ASC = 12
SORT_BY_LEXICOGRAPHIC_NAMESPACE_DESC = 13
SORT_BY_INCIDENCE_NAMESPACE_ASC = 14
SORT_BY_INCIDENCE_NAMESPACE_DESC = 15
SORT_BY_LEXICOGRAPHIC_IGNORE_NAMESPACE_ASC = 16
SORT_BY_LEXICOGRAPHIC_IGNORE_NAMESPACE_DESC = 17
old_default_tag_sort = old_options[ 'default_tag_sort' ]
from hydrus.client.metadata import ClientTagSorting
sort_type = ClientTagSorting.SORT_BY_HUMAN_TAG
if old_default_tag_sort in ( SORT_BY_LEXICOGRAPHIC_ASC, SORT_BY_LEXICOGRAPHIC_DESC, SORT_BY_LEXICOGRAPHIC_NAMESPACE_ASC, SORT_BY_LEXICOGRAPHIC_NAMESPACE_ASC ):
sort_type = ClientTagSorting.SORT_BY_HUMAN_TAG
elif old_default_tag_sort in ( SORT_BY_LEXICOGRAPHIC_IGNORE_NAMESPACE_ASC, SORT_BY_LEXICOGRAPHIC_IGNORE_NAMESPACE_DESC ):
sort_type = ClientTagSorting.SORT_BY_HUMAN_SUBTAG
elif old_default_tag_sort in ( SORT_BY_INCIDENCE_ASC, SORT_BY_INCIDENCE_DESC, SORT_BY_INCIDENCE_NAMESPACE_ASC, SORT_BY_INCIDENCE_NAMESPACE_DESC ):
sort_type = ClientTagSorting.SORT_BY_COUNT
if old_default_tag_sort in ( SORT_BY_INCIDENCE_ASC, SORT_BY_INCIDENCE_NAMESPACE_ASC, SORT_BY_LEXICOGRAPHIC_ASC, SORT_BY_LEXICOGRAPHIC_IGNORE_NAMESPACE_ASC, SORT_BY_LEXICOGRAPHIC_NAMESPACE_ASC ):
sort_order = CC.SORT_ASC
else:
sort_order = CC.SORT_DESC
use_siblings = True
if old_default_tag_sort in ( SORT_BY_INCIDENCE_NAMESPACE_ASC, SORT_BY_INCIDENCE_NAMESPACE_DESC, SORT_BY_LEXICOGRAPHIC_NAMESPACE_ASC, SORT_BY_LEXICOGRAPHIC_NAMESPACE_DESC ):
group_by = ClientTagSorting.GROUP_BY_NAMESPACE
else:
group_by = ClientTagSorting.GROUP_BY_NOTHING
tag_sort = ClientTagSorting.TagSort(
sort_type = sort_type,
sort_order = sort_order,
use_siblings = use_siblings,
group_by = group_by
)
new_options.SetDefaultTagSort( tag_sort )
self.modules_serialisable.SetJSONDump( new_options )
except Exception as e:
HydrusData.PrintException( e )
message = 'Trying to convert your old default tag sort to the new format failed! Please set it again in the options.'
self.pub_initial_message( message )
if version == 432:
try:
domain_manager = self.modules_serialisable.GetJSONDump( HydrusSerialisable.SERIALISABLE_TYPE_NETWORK_DOMAIN_MANAGER )
domain_manager.Initialise()
#
domain_manager.OverwriteDefaultGUGs( [
'twitter syndication profile lookup (limited) (with replies)',
'twitter syndication profile lookup (limited)'
] )
#
domain_manager.OverwriteDefaultURLClasses( [
'twitter syndication api profile',
'twitter syndication api tweet',
'twitter tweet'
] )
#
domain_manager.OverwriteDefaultParsers( [
'twitter syndication api profile parser',
'twitter syndication api tweet parser'
] )
#
domain_manager.TryToLinkURLClassesAndParsers()
#
self.modules_serialisable.SetJSONDump( domain_manager )
except Exception as e:
HydrusData.PrintException( e )
message = 'Trying to add the twitter downloader failed! Please let hydrus dev know!'
self.pub_initial_message( message )
if version == 435:
try:
self._RegenerateTagPendingMappingsCache()
types_to_delete = (
HC.SERVICE_INFO_NUM_PENDING_MAPPINGS,
HC.SERVICE_INFO_NUM_PENDING_TAG_SIBLINGS,
HC.SERVICE_INFO_NUM_PENDING_TAG_PARENTS,
HC.SERVICE_INFO_NUM_PETITIONED_MAPPINGS,
HC.SERVICE_INFO_NUM_PETITIONED_TAG_SIBLINGS,
HC.SERVICE_INFO_NUM_PETITIONED_TAG_PARENTS,
HC.SERVICE_INFO_NUM_PENDING_FILES,
HC.SERVICE_INFO_NUM_PETITIONED_FILES
)
self._DeleteServiceInfo( types_to_delete = types_to_delete )
except Exception as e:
HydrusData.PrintException( e )
message = 'Trying to regenerate the pending tag cache failed! This is not a big deal, but you might still have a bad pending count for your pending menu. Error information has been written to the log. Please let hydrus dev know!'
self.pub_initial_message( message )
if version == 436:
result = self._Execute( 'SELECT sql FROM sqlite_master WHERE name = ?;', ( 'deleted_files', ) ).fetchone()
if result is None:
raise Exception( 'No deleted_files table!!!' )
( s, ) = result
if 'timestamp' not in s:
self._Execute( 'ALTER TABLE deleted_files ADD COLUMN timestamp INTEGER;' )
self._Execute( 'ALTER TABLE deleted_files ADD COLUMN original_timestamp INTEGER;' )
self._Execute( 'UPDATE deleted_files SET timestamp = ?, original_timestamp = ?;', ( None, None ) )
my_files_service_id = self.modules_services.GetServiceId( CC.LOCAL_FILE_SERVICE_KEY )
self._Execute( 'INSERT OR IGNORE INTO deleted_files ( service_id, hash_id, timestamp, original_timestamp ) SELECT ?, hash_id, timestamp, original_timestamp FROM deleted_files WHERE service_id = ?;', ( my_files_service_id, self.modules_services.combined_local_file_service_id ) )
self._Execute( 'INSERT OR IGNORE INTO deleted_files ( service_id, hash_id, timestamp, original_timestamp ) SELECT ?, hash_id, ?, timestamp FROM current_files WHERE service_id = ?;', ( my_files_service_id, None, self.modules_services.trash_service_id ) )
self._CreateIndex( 'deleted_files', [ 'timestamp' ] )
self._CreateIndex( 'deleted_files', [ 'original_timestamp' ] )
self._Execute( 'DELETE FROM service_info WHERE info_type = ?;', ( HC.SERVICE_INFO_NUM_DELETED_FILES, ) )
self.modules_db_maintenance.AnalyzeTable( 'deleted_files' )
if version == 438:
try:
domain_manager = self.modules_serialisable.GetJSONDump( HydrusSerialisable.SERIALISABLE_TYPE_NETWORK_DOMAIN_MANAGER )
domain_manager.Initialise()
#
domain_manager.OverwriteDefaultURLClasses( ( 'imgur single media file url', ) )
#
self.modules_serialisable.SetJSONDump( domain_manager )
except Exception as e:
HydrusData.PrintException( e )
message = 'Trying to update some url classes failed! Please let hydrus dev know!'
self.pub_initial_message( message )
if version == 440:
try:

9
hydrus/client/gui/ClientGUI.py
View File

@ -1097,7 +1097,9 @@ class FrameGUI( ClientGUITopLevelWindows.MainFrameThatResizes, CAC.ApplicationCo
text = 'DO NOT RUN THIS UNLESS YOU KNOW YOU NEED TO'
text += os.linesep * 2
text += 'This will instruct the database to review its file records and delete any orphans. You typically do not ever see these files and they are basically harmless, but they can offset some file counts confusingly. You probably only need to run this if you can\'t process the apparent last handful of duplicate filter pairs or hydrus dev otherwise told you to try it.'
text += 'This will instruct the database to review its file records\' integrity. If anything appears to be in a specific domain (e.g. my files) but not an umbrella domain (e.g. all my files), or vice versa, it will be removed.'
text += os.linesep * 2
text += 'You typically do not ever see these files and they are basically harmless, but they can offset some file counts confusingly and may break other maintenance routines. You probably only need to run this if you can\'t process the apparent last handful of duplicate filter pairs or hydrus dev otherwise told you to try it.'
text += os.linesep * 2
text += 'It will create a popup message while it works and inform you of the number of orphan records found.'
@ -3020,6 +3022,10 @@ class FrameGUI( ClientGUITopLevelWindows.MainFrameThatResizes, CAC.ApplicationCo
ClientGUIMenus.AppendMenuCheckItem( file_maintenance_menu, 'work file jobs during normal time', 'Control whether file maintenance can work during normal time.', current_value, func )
ClientGUIMenus.AppendSeparator( file_maintenance_menu )
ClientGUIMenus.AppendMenuItem( file_maintenance_menu, 'clear orphan files', 'Clear out surplus files that have found their way into the file structure.', self._ClearOrphanFiles )
ClientGUIMenus.AppendMenu( menu, file_maintenance_menu, 'file maintenance' )
maintenance_submenu = QW.QMenu( menu )
@ -3029,7 +3035,6 @@ class FrameGUI( ClientGUITopLevelWindows.MainFrameThatResizes, CAC.ApplicationCo
ClientGUIMenus.AppendSeparator( maintenance_submenu )
ClientGUIMenus.AppendMenuItem( maintenance_submenu, 'clear orphan files', 'Clear out surplus files that have found their way into the file structure.', self._ClearOrphanFiles )
ClientGUIMenus.AppendMenuItem( maintenance_submenu, 'clear orphan file records', 'Clear out surplus file records that have not been deleted correctly.', self._ClearOrphanFileRecords )
ClientGUIMenus.AppendMenuItem( maintenance_submenu, 'clear orphan tables', 'Clear out surplus db tables that have not been deleted correctly.', self._ClearOrphanTables )

106
hydrus/client/gui/ClientGUIScrolledPanelsReview.py
View File

@ -2756,7 +2756,7 @@ class ReviewHowBonedAmI( ClientGUIScrolledPanels.ReviewPanel ):
QP.AddToLayout( vbox, win, CC.FLAGS_CENTER )
if num_supertotal == 0:
if num_total == 0:
nothing_label = 'You have yet to board the ride.'
@ -2766,6 +2766,48 @@ class ReviewHowBonedAmI( ClientGUIScrolledPanels.ReviewPanel ):
else:
supertotal_average_filesize = size_supertotal // num_supertotal
current_num_percent = num_total / num_supertotal
current_size_percent = size_total / size_supertotal
current_average_filesize = size_total // num_total
inbox_num_percent = num_inbox / num_total
inbox_size_percent = size_inbox / size_total
if num_inbox > 0:
inbox_average_filesize = size_inbox // num_inbox
else:
inbox_average_filesize = 0
archive_num_percent = num_archive / num_total
archive_size_percent = size_archive / size_total
if num_archive > 0:
archive_average_filesize = size_archive // num_archive
else:
archive_average_filesize = 0
deleted_num_percent = num_deleted / num_supertotal
deleted_size_percent = size_deleted / size_supertotal
if num_deleted > 0:
deleted_average_filesize = size_deleted // num_deleted
else:
deleted_average_filesize = 0
notebook = ClientGUICommon.BetterNotebook( self )
#
@ -2774,41 +2816,65 @@ class ReviewHowBonedAmI( ClientGUIScrolledPanels.ReviewPanel ):
panel_vbox = QP.VBoxLayout()
average_filesize = size_total // num_total
# spacing to make the weird unicode characters join up neater
text_table_layout = QP.GridLayout( cols = 4, spacing = 0 )
summary_label = 'Total: {} files, totalling {}, averaging {}'.format( HydrusData.ToHumanInt( num_total ), HydrusData.ToHumanBytes( size_total ), HydrusData.ToHumanBytes( average_filesize ) )
text_table_layout.setHorizontalSpacing( ClientGUIFunctions.ConvertTextToPixelWidth( self, 2 ) )
summary_st = ClientGUICommon.BetterStaticText( panel, label = summary_label )
text_table_layout.setColumnStretch( 0, 1 )
QP.AddToLayout( panel_vbox, summary_st, CC.FLAGS_CENTER )
#
num_archive_percent = num_archive / num_total
size_archive_percent = size_archive / size_total
QP.AddToLayout( text_table_layout, QW.QWidget( panel ), CC.FLAGS_ON_LEFT )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = 'Files' ), CC.FLAGS_CENTER )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = 'Size' ), CC.FLAGS_CENTER )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = 'Average' ), CC.FLAGS_CENTER )
num_inbox_percent = num_inbox / num_total
size_inbox_percent = size_inbox / size_total
#
num_deleted_percent = num_deleted / num_supertotal
size_deleted_percent = size_deleted / size_supertotal
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = 'Total Ever Imported:' ), CC.FLAGS_ON_LEFT )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = HydrusData.ToHumanInt( num_supertotal ) ), CC.FLAGS_ON_RIGHT )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = HydrusData.ToHumanBytes( size_supertotal ) ), CC.FLAGS_ON_RIGHT )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = HydrusData.ToHumanBytes( supertotal_average_filesize ) ), CC.FLAGS_ON_RIGHT )
archive_label = 'Archive: {} files ({}), totalling {} ({})'.format( HydrusData.ToHumanInt( num_archive ), ClientData.ConvertZoomToPercentage( num_archive_percent ), HydrusData.ToHumanBytes( size_archive ), ClientData.ConvertZoomToPercentage( size_archive_percent ) )
#
archive_st = ClientGUICommon.BetterStaticText( panel, label = archive_label )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = '\u251cAll My Files:' ), CC.FLAGS_ON_LEFT )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = '{} ({})'.format( HydrusData.ToHumanInt( num_total ), ClientData.ConvertZoomToPercentage( current_num_percent ) ) ), CC.FLAGS_ON_RIGHT )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = '{} ({})'.format( HydrusData.ToHumanBytes( size_total ), ClientData.ConvertZoomToPercentage( current_size_percent ) ) ), CC.FLAGS_ON_RIGHT )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = HydrusData.ToHumanBytes( current_average_filesize ) ), CC.FLAGS_ON_RIGHT )
inbox_label = 'Inbox: {} files ({}), totalling {} ({})'.format( HydrusData.ToHumanInt( num_inbox ), ClientData.ConvertZoomToPercentage( num_inbox_percent ), HydrusData.ToHumanBytes( size_inbox ), ClientData.ConvertZoomToPercentage( size_inbox_percent ) )
#
inbox_st = ClientGUICommon.BetterStaticText( panel, label = inbox_label )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = '\u2502\u251cInbox:' ), CC.FLAGS_ON_LEFT )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = '{} ({})'.format( HydrusData.ToHumanInt( num_inbox ), ClientData.ConvertZoomToPercentage( inbox_num_percent ) ) ), CC.FLAGS_ON_RIGHT )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = '{} ({})'.format( HydrusData.ToHumanBytes( size_inbox ), ClientData.ConvertZoomToPercentage( inbox_size_percent ) ) ), CC.FLAGS_ON_RIGHT )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = HydrusData.ToHumanBytes( inbox_average_filesize ) ), CC.FLAGS_ON_RIGHT )
deleted_label = 'Deleted: {} files ({}), totalling {} ({})'.format( HydrusData.ToHumanInt( num_deleted ), ClientData.ConvertZoomToPercentage( num_deleted_percent ), HydrusData.ToHumanBytes( size_deleted ), ClientData.ConvertZoomToPercentage( size_deleted_percent ) )
#
deleted_st = ClientGUICommon.BetterStaticText( panel, label = deleted_label )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = '\u2502\u2514Archive:' ), CC.FLAGS_ON_LEFT )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = '{} ({})'.format( HydrusData.ToHumanInt( num_archive ), ClientData.ConvertZoomToPercentage( archive_num_percent ) ) ), CC.FLAGS_ON_RIGHT )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = '{} ({})'.format( HydrusData.ToHumanBytes( size_archive ), ClientData.ConvertZoomToPercentage( archive_size_percent ) ) ), CC.FLAGS_ON_RIGHT )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = HydrusData.ToHumanBytes( archive_average_filesize ) ), CC.FLAGS_ON_RIGHT )
QP.AddToLayout( panel_vbox, archive_st, CC.FLAGS_CENTER )
QP.AddToLayout( panel_vbox, inbox_st, CC.FLAGS_CENTER )
QP.AddToLayout( panel_vbox, deleted_st, CC.FLAGS_CENTER )
#
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = '\u2514Deleted:' ), CC.FLAGS_ON_LEFT )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = '{} ({})'.format( HydrusData.ToHumanInt( num_deleted ), ClientData.ConvertZoomToPercentage( deleted_num_percent ) ) ), CC.FLAGS_ON_RIGHT )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = '{} ({})'.format( HydrusData.ToHumanBytes( size_deleted ), ClientData.ConvertZoomToPercentage( deleted_size_percent ) ) ), CC.FLAGS_ON_RIGHT )
QP.AddToLayout( text_table_layout, ClientGUICommon.BetterStaticText( panel, label = HydrusData.ToHumanBytes( deleted_average_filesize ) ), CC.FLAGS_ON_RIGHT )
#
QP.AddToLayout( panel_vbox, text_table_layout, CC.FLAGS_EXPAND_PERPENDICULAR )
#
if 'earliest_import_time' in boned_stats:
panel_vbox.addSpacing( ClientGUIFunctions.ConvertTextToPixelWidth( self, 2 ) )
eit = boned_stats[ 'earliest_import_time' ]
eit_label = 'Earliest file import: {} ({})'.format( HydrusData.ConvertTimestampToPrettyTime( eit ), HydrusData.TimestampToPrettyTimeDelta( eit ) )

66
hydrus/client/gui/ClientGUIShortcutControls.py
View File

@ -11,7 +11,6 @@ from hydrus.core import HydrusSerialisable
from hydrus.client import ClientApplicationCommand as CAC
from hydrus.client import ClientConstants as CC
from hydrus.client import ClientData
from hydrus.client.gui import ClientGUIDialogsQuick
from hydrus.client.gui import ClientGUIScrolledPanels
from hydrus.client.gui import ClientGUIShortcuts
@ -109,7 +108,11 @@ class EditShortcutSetPanel( ClientGUIScrolledPanels.EditPanel ):
self._shortcuts_panel.SetListCtrl( self._shortcuts )
self._shortcuts_panel.AddImportExportButtons( ( ClientGUIShortcuts.ShortcutSet, ), self._AddShortcutSet, custom_get_callable = self._GetSelectedShortcutSet )
self._shortcuts_panel.AddImportExportButtons( ( ClientGUIShortcuts.ShortcutSet, ), self._AddShortcutSet, custom_get_callable = self._GetSelectedShortcutSet, and_duplicate_button = False )
tt = 'Click this to replicate the current selection of commands with "incremented" shortcuts. If you want to create a list of "set rating to 1, 2, 3" or "set tag" commands, use this.'
self._shortcuts_panel.AddButton( 'special duplicate', self._SpecialDuplicate, enabled_only_on_selection = True, tooltip = tt )
self._shortcuts.setMinimumSize( QC.QSize( 360, 480 ) )
@ -137,7 +140,7 @@ class EditShortcutSetPanel( ClientGUIScrolledPanels.EditPanel ):
self._name.setEnabled( False )
self._shortcuts.AddDatas( shortcuts )
self._shortcuts.AddDatas( shortcuts.GetShortcutsAndCommands() )
self._shortcuts.Sort()
@ -151,6 +154,14 @@ class EditShortcutSetPanel( ClientGUIScrolledPanels.EditPanel ):
vbox = QP.VBoxLayout()
message = 'Please note the shortcut system does not support multiple commands per shortcut yet. If there are shortcut duplicates in this list, only one command will ever fire.'
st = ClientGUICommon.BetterStaticText( self, label = message )
st.setWordWrap( True )
QP.AddToLayout( vbox, st, CC.FLAGS_EXPAND_PERPENDICULAR )
QP.AddToLayout( vbox, ClientGUICommon.WrapInText( self._name, self, 'name: ' ), CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
if name in ClientGUIShortcuts.shortcut_names_to_descriptions:
@ -182,10 +193,10 @@ class EditShortcutSetPanel( ClientGUIScrolledPanels.EditPanel ):
def _AddShortcutSet( self, shortcut_set: ClientGUIShortcuts.ShortcutSet ):
self._shortcuts.AddDatas( shortcut_set )
self._shortcuts.AddDatas( shortcut_set.GetShortcutsAndCommands() )
def _GetSelectedShortcutSet( self ):
def _GetSelectedShortcutSet( self ) -> ClientGUIShortcuts.ShortcutSet:
name = self._name.text()
@ -199,6 +210,51 @@ class EditShortcutSetPanel( ClientGUIScrolledPanels.EditPanel ):
return shortcut_set
def _SpecialDuplicate( self ):
all_existing_shortcuts = { shortcut for ( shortcut, command ) in self._shortcuts.GetData() }
shortcut_set = self._GetSelectedShortcutSet()
num_not_added = 0
for ( shortcut, command ) in shortcut_set.GetShortcutsAndCommands():
addee_shortcut = shortcut.Duplicate()
command = command.Duplicate()
while addee_shortcut in all_existing_shortcuts:
try:
addee_shortcut.TryToIncrementKey()
except HydrusExceptions.VetoException:
num_not_added += 1
break
if addee_shortcut not in all_existing_shortcuts:
self._shortcuts.AddDatas( [ ( addee_shortcut, command ) ] )
all_existing_shortcuts.add( addee_shortcut )
self._shortcuts.Sort()
if num_not_added > 0:
message = 'Not every shortcut could find a new key to use, sorry!'
QW.QMessageBox.information( self, 'Information', message )
def AddShortcut( self ):
shortcut = ClientGUIShortcuts.Shortcut()

34
hydrus/client/gui/ClientGUIShortcuts.py
View File

@ -6,6 +6,7 @@ from qtpy import QtGui as QG
from hydrus.core import HydrusConstants as HC
from hydrus.core import HydrusData
from hydrus.core import HydrusExceptions
from hydrus.core import HydrusGlobals as HG
from hydrus.core import HydrusSerialisable
@ -931,6 +932,34 @@ class Shortcut( HydrusSerialisable.SerialisableBase ):
return s
def TryToIncrementKey( self ):
if self.shortcut_type == SHORTCUT_TYPE_KEYBOARD_CHARACTER:
new_shortcut_key = self.shortcut_key + 1
if ClientData.OrdIsAlphaLower( new_shortcut_key ) or ClientData.OrdIsNumber( new_shortcut_key ):
self.shortcut_key = new_shortcut_key
return
elif self.shortcut_type == SHORTCUT_TYPE_KEYBOARD_SPECIAL:
new_shortcut_key = self.shortcut_key + 1
if new_shortcut_key in special_key_shortcut_str_lookup:
self.shortcut_key = new_shortcut_key
return
raise HydrusExceptions.VetoException( 'Sorry, cannot increment that shortcut!' )
HydrusSerialisable.SERIALISABLE_TYPES_TO_OBJECT_TYPES[ HydrusSerialisable.SERIALISABLE_TYPE_SHORTCUT ] = Shortcut
class ShortcutSet( HydrusSerialisable.SerialisableBaseNamed ):
@ -1076,6 +1105,11 @@ class ShortcutSet( HydrusSerialisable.SerialisableBaseNamed ):
return shortcuts
def GetShortcutsAndCommands( self ):
return list( self )
def SetCommand( self, shortcut, command ):
self._shortcuts_to_commands[ shortcut ] = command

13
hydrus/client/gui/canvas/ClientGUICanvas.py
View File

@ -2362,8 +2362,6 @@ class CanvasWithDetails( Canvas ):
original_pen = painter.pen()
painter.setFont( QW.QApplication.font() )
tags_manager = self._current_media.GetTagsManager()
current = tags_manager.GetCurrent( CC.COMBINED_TAG_SERVICE_KEY, ClientTags.TAG_DISPLAY_SINGLE_MEDIA )
@ -4254,9 +4252,18 @@ class CanvasMediaListFilterArchiveDelete( CanvasMediaList ):
location_contexts_to_present_options_for.extend( [ ClientLocation.LocationContext.STATICCreateSimple( service_key ) for service_key in local_file_domain_service_keys ] )
all_my_files_location_context = ClientLocation.LocationContext.STATICCreateSimple( CC.COMBINED_LOCAL_MEDIA_SERVICE_KEY )
if len( local_file_domain_service_keys ) > 1:
location_contexts_to_present_options_for.append( ClientLocation.LocationContext.STATICCreateSimple( CC.COMBINED_LOCAL_MEDIA_SERVICE_KEY ) )
location_contexts_to_present_options_for.append( all_my_files_location_context )
elif len( local_file_domain_service_keys ) == 1:
if all_my_files_location_context in location_contexts_to_present_options_for:
location_contexts_to_present_options_for.remove( all_my_files_location_context )
specific_local_service_keys = [ service_key for service_key in current_local_service_keys if HG.client_controller.services_manager.GetServiceType( service_key ) in HC.SPECIFIC_LOCAL_FILE_SERVICES ]

4
hydrus/client/gui/lists/ClientGUIListBoxes.py
View File

@ -964,10 +964,6 @@ class ListBox( QW.QScrollArea ):
self._async_text_info_shared_data = dict()
self._async_text_info_updater = self._InitialiseAsyncTextInfoUpdater()
#
self.setFont( QW.QApplication.font() )
def __len__( self ):

15
hydrus/client/gui/lists/ClientGUIListCtrl.py
View File

@ -1428,10 +1428,15 @@ class BetterListCtrlPanel( QW.QWidget ):
self._UpdateButtons()
def AddButton( self, label, clicked_func, enabled_only_on_selection = False, enabled_only_on_single_selection = False, enabled_check_func = None ):
def AddButton( self, label, clicked_func, enabled_only_on_selection = False, enabled_only_on_single_selection = False, enabled_check_func = None, tooltip = None ):
button = ClientGUICommon.BetterButton( self, label, clicked_func )
if tooltip is not None:
button.setToolTip( tooltip )
self._AddButton( button, enabled_only_on_selection = enabled_only_on_selection, enabled_only_on_single_selection = enabled_only_on_single_selection, enabled_check_func = enabled_check_func )
self._UpdateButtons()
@ -1464,7 +1469,7 @@ class BetterListCtrlPanel( QW.QWidget ):
self.AddButton( 'delete', self._listctrl.ProcessDeleteAction, enabled_check_func = enabled_check_func, enabled_only_on_selection = enabled_only_on_selection )
def AddImportExportButtons( self, permitted_object_types, import_add_callable, custom_get_callable = None ):
def AddImportExportButtons( self, permitted_object_types, import_add_callable, custom_get_callable = None, and_duplicate_button = True ):
self._permitted_object_types = permitted_object_types
self._import_add_callable = import_add_callable
@ -1494,7 +1499,11 @@ class BetterListCtrlPanel( QW.QWidget ):
self.AddMenuButton( 'export', export_menu_items, enabled_only_on_selection = True )
self.AddMenuButton( 'import', import_menu_items )
self.AddButton( 'duplicate', self._Duplicate, enabled_only_on_selection = True )
if and_duplicate_button:
self.AddButton( 'duplicate', self._Duplicate, enabled_only_on_selection = True )
self.setAcceptDrops( True )
self.installEventFilter( ClientGUIDragDrop.FileDropTarget( self, filenames_callable = self.ImportFromDragDrop ) )

66
hydrus/client/gui/pages/ClientGUIResults.py
View File

@ -44,6 +44,7 @@ from hydrus.client.gui.canvas import ClientGUICanvasFrame
from hydrus.client.gui.networking import ClientGUIHydrusNetwork
from hydrus.client.media import ClientMedia
from hydrus.client.metadata import ClientTags
from hydrus.client.gui.search import ClientGUILocation
def AddDuplicatesMenu( win: QW.QWidget, menu: QW.QMenu, location_context: ClientLocation.LocationContext, focus_singleton: ClientMedia.Media, num_selected: int, collections_selected: bool ):
@ -1587,6 +1588,10 @@ class MediaPanel( ClientMedia.ListeningMediaList, QW.QScrollArea, CAC.Applicatio
message = 'This will regenerate the {} selected files\' thumbnails, but only if they are the wrong size.'.format( HydrusData.ToHumanInt( num_files ) )
else:
message = ClientFiles.regen_file_enum_to_description_lookup[ job_type ]
do_it_now = True
@ -3909,6 +3914,8 @@ class MediaPanelThumbnails( MediaPanel ):
# valid commands for the files
current_file_service_keys = set()
uploadable_file_service_keys = set()
downloadable_file_service_keys = set()
@ -3932,6 +3939,10 @@ class MediaPanelThumbnails( MediaPanel ):
pending = locations_manager.GetPending()
petitioned = locations_manager.GetPetitioned()
# ALL
current_file_service_keys.update( current )
# FILE REPOS
# we can upload (set pending) to a repo_id when we have permission, a file is local, not current, not pending, and either ( not deleted or we_can_overrule )
@ -4129,7 +4140,9 @@ class MediaPanelThumbnails( MediaPanel ):
ClientGUIMenus.AppendSeparator( menu )
for file_service_key in all_local_file_domains_sorted:
local_file_service_keys_we_are_in = sorted( current_file_service_keys.intersection( local_media_file_service_keys ), key = HG.client_controller.services_manager.GetName )
for file_service_key in local_file_service_keys_we_are_in:
ClientGUIMenus.AppendMenuItem( menu, '{} from {}'.format( local_delete_phrase, HG.client_controller.services_manager.GetName( file_service_key ) ), 'Delete the selected files.', self._Delete, file_service_key )
@ -4180,6 +4193,10 @@ class MediaPanelThumbnails( MediaPanel ):
ClientGUIMenus.AppendMenuItem( regen_menu, 'thumbnails, but only if wrong size', 'Regenerate the selected files\' thumbnails, but only if they are the wrong size.', self._RegenerateFileData, ClientFiles.REGENERATE_FILE_DATA_JOB_REFIT_THUMBNAIL )
ClientGUIMenus.AppendMenuItem( regen_menu, 'thumbnails', 'Regenerate the selected files\'s thumbnails.', self._RegenerateFileData, ClientFiles.REGENERATE_FILE_DATA_JOB_FORCE_THUMBNAIL )
ClientGUIMenus.AppendMenuItem( regen_menu, 'file metadata', 'Regenerated the selected files\' metadata and thumbnails.', self._RegenerateFileData, ClientFiles.REGENERATE_FILE_DATA_JOB_FILE_METADATA )
ClientGUIMenus.AppendMenuItem( regen_menu, 'similar files data', 'Regenerated the selected files\' perceptual hashes.', self._RegenerateFileData, ClientFiles.REGENERATE_FILE_DATA_JOB_SIMILAR_FILES_METADATA )
ClientGUIMenus.AppendMenuItem( regen_menu, 'duplicate pixel data', 'Regenerated the selected files\' pixel hashes.', self._RegenerateFileData, ClientFiles.REGENERATE_FILE_DATA_JOB_PIXEL_HASH )
ClientGUIMenus.AppendMenuItem( regen_menu, 'full presence check', 'Check file presence and try to fix.', self._RegenerateFileData, ClientFiles.REGENERATE_FILE_DATA_JOB_FILE_INTEGRITY_PRESENCE_TRY_URL_ELSE_REMOVE_RECORD )
ClientGUIMenus.AppendMenuItem( regen_menu, 'full integrity check', 'Check file integrity and try to fix.', self._RegenerateFileData, ClientFiles.REGENERATE_FILE_DATA_JOB_FILE_INTEGRITY_DATA_TRY_URL_ELSE_REMOVE_RECORD )
ClientGUIMenus.AppendMenu( manage_menu, regen_menu, 'regenerate' )
@ -4667,24 +4684,9 @@ def AddRemoveMenu( win: MediaPanel, menu, filter_counts, all_specific_file_domai
ClientGUIMenus.AppendSeparator( remove_menu )
all_specific_file_domains = list( all_specific_file_domains )
all_specific_file_domains = ClientLocation.SortFileServiceKeysNicely( all_specific_file_domains )
if CC.TRASH_SERVICE_KEY in all_specific_file_domains:
all_specific_file_domains.remove( CC.TRASH_SERVICE_KEY )
all_specific_file_domains.insert( 0, CC.TRASH_SERVICE_KEY )
for service in HG.client_controller.services_manager.GetLocalMediaFileServices():
service_key = service.GetServiceKey()
if service_key in all_specific_file_domains:
all_specific_file_domains.remove( service_key )
all_specific_file_domains.insert( 0, service_key )
all_specific_file_domains = ClientLocation.FilterOutRedundantMetaServices( all_specific_file_domains )
for file_service_key in all_specific_file_domains:
@ -4757,24 +4759,9 @@ def AddSelectMenu( win: MediaPanel, menu, filter_counts, all_specific_file_domai
ClientGUIMenus.AppendSeparator( select_menu )
all_specific_file_domains = list( all_specific_file_domains )
all_specific_file_domains = ClientLocation.SortFileServiceKeysNicely( all_specific_file_domains )
if CC.TRASH_SERVICE_KEY in all_specific_file_domains:
all_specific_file_domains.remove( CC.TRASH_SERVICE_KEY )
all_specific_file_domains.insert( 0, CC.TRASH_SERVICE_KEY )
for service in HG.client_controller.services_manager.GetLocalMediaFileServices():
service_key = service.GetServiceKey()
if service_key in all_specific_file_domains:
all_specific_file_domains.remove( service_key )
all_specific_file_domains.insert( 0, service_key )
all_specific_file_domains = ClientLocation.FilterOutRedundantMetaServices( all_specific_file_domains )
for file_service_key in all_specific_file_domains:
@ -4880,6 +4867,9 @@ class Thumbnail( Selectable ):
# this isn't a Qt object, we need to set the font explitly to get font size changes from QSS etc..
painter.setFont( HG.client_controller.gui.font() )
painter.setPen( QC.Qt.NoPen )
painter.setBrush( QG.QBrush( new_options.GetColour( background_colour_type ) ) )
@ -4930,8 +4920,6 @@ class Thumbnail( Selectable ):
text_colour_with_alpha = upper_tag_summary_generator.GetTextColour()
painter.setFont( QW.QApplication.font() )
background_colour_with_alpha = upper_tag_summary_generator.GetBackgroundColour()
painter.setBrush( QG.QBrush( background_colour_with_alpha ) )
@ -4959,8 +4947,6 @@ class Thumbnail( Selectable ):
text_colour_with_alpha = lower_tag_summary_generator.GetTextColour()
painter.setFont( QW.QApplication.font() )
background_colour_with_alpha = lower_tag_summary_generator.GetBackgroundColour()
painter.setBrush( QG.QBrush( background_colour_with_alpha ) )
@ -5089,8 +5075,6 @@ class Thumbnail( Selectable ):
num_files_str = HydrusData.ToHumanInt( self.GetNumFiles() )
painter.setFont( QW.QApplication.font() )
( text_size, num_files_str ) = ClientGUIFunctions.GetTextSizeFromPainter( painter, num_files_str )
text_width = text_size.width()

7
hydrus/client/gui/search/ClientGUIACDropdown.py
View File

@ -1459,6 +1459,13 @@ class AutoCompleteDropdownTags( AutoCompleteDropdown ):
self._RestoreTextCtrlFocus()
if location_context.IsAllKnownFiles() and self._tag_service_key == CC.COMBINED_TAG_SERVICE_KEY:
top_local_tag_service_key = list( HG.client_controller.services_manager.GetServiceKeys( ( HC.LOCAL_TAG, ) ) )[0]
self._SetTagService( top_local_tag_service_key )
self.locationChanged.emit( location_context )
self._SetListDirty()

47
hydrus/client/gui/search/ClientGUILocation.py
View File

@ -1,3 +1,5 @@
import typing
from qtpy import QtCore as QC
from qtpy import QtWidgets as QW
@ -13,47 +15,6 @@ from hydrus.client.gui import ClientGUITopLevelWindowsPanels
from hydrus.client.gui import QtPorting as QP
from hydrus.client.gui.widgets import ClientGUICommon
def GetPossibleFileDomainServicesInOrder( all_known_files_allowed: bool, only_local_file_domains_allowed: bool ):
services_manager = HG.client_controller.services_manager
service_types_in_order = [ HC.LOCAL_FILE_DOMAIN ]
if not only_local_file_domains_allowed:
advanced_mode = HG.client_controller.new_options.GetBoolean( 'advanced_mode' )
if len( services_manager.GetServices( ( HC.LOCAL_FILE_DOMAIN, ) ) ) > 1 or advanced_mode:
service_types_in_order.append( HC.COMBINED_LOCAL_MEDIA )
service_types_in_order.append( HC.LOCAL_FILE_TRASH_DOMAIN )
if advanced_mode:
service_types_in_order.append( HC.LOCAL_FILE_UPDATE_DOMAIN )
if advanced_mode:
service_types_in_order.append( HC.COMBINED_LOCAL_FILE )
service_types_in_order.append( HC.FILE_REPOSITORY )
service_types_in_order.append( HC.IPFS )
if all_known_files_allowed:
service_types_in_order.append( HC.COMBINED_FILE )
services = services_manager.GetServices( service_types_in_order )
return services
class EditMultipleLocationContextPanel( ClientGUIScrolledPanels.EditPanel ):
def __init__( self, parent: QW.QWidget, location_context: ClientLocation.LocationContext, all_known_files_allowed: bool, only_local_file_domains_allowed: bool ):
@ -66,7 +27,7 @@ class EditMultipleLocationContextPanel( ClientGUIScrolledPanels.EditPanel ):
self._location_list = ClientGUICommon.BetterCheckBoxList( self )
services = GetPossibleFileDomainServicesInOrder( all_known_files_allowed, only_local_file_domains_allowed )
services = ClientLocation.GetPossibleFileDomainServicesInOrder( all_known_files_allowed, only_local_file_domains_allowed )
for service in services:
@ -179,7 +140,7 @@ class LocationSearchContextButton( ClientGUICommon.BetterButton ):
def _EditLocation( self ):
services = GetPossibleFileDomainServicesInOrder( self._IsAllKnownFilesServiceTypeAllowed(), self._only_importable_domains_allowed )
services = ClientLocation.GetPossibleFileDomainServicesInOrder( self._IsAllKnownFilesServiceTypeAllowed(), self._only_importable_domains_allowed )
menu = QW.QMenu()

2
hydrus/client/gui/widgets/ClientGUICommon.py
View File

@ -1857,7 +1857,7 @@ class StaticBox( QW.QFrame ):
self._sizer = QP.VBoxLayout()
normal_font = QW.QApplication.font()
normal_font = self.font()
normal_font_size = normal_font.pointSize()
normal_font_family = normal_font.family()

6
hydrus/client/importing/ClientImportLocal.py
View File

@ -155,7 +155,7 @@ class HDDImport( HydrusSerialisable.SerialisableBase ):
with self._lock:
self._file_status = ClientImportControl.NeatenStatusText( text )
self._files_status = ClientImportControl.NeatenStatusText( text )
@ -355,7 +355,7 @@ class HDDImport( HydrusSerialisable.SerialisableBase ):
with self._lock:
self._file_status = str( e )
self._files_status = str( e )
break
@ -371,7 +371,7 @@ class HDDImport( HydrusSerialisable.SerialisableBase ):
with self._lock:
self._file_status = 'stopping work: {}'.format( str( e ) )
self._files_status = 'stopping work: {}'.format( str( e ) )
HydrusData.ShowException( e )

21
hydrus/client/media/ClientMediaManagers.py
View File

@ -15,23 +15,30 @@ from hydrus.client import ClientSearch
from hydrus.client import ClientTime
from hydrus.client.metadata import ClientTags
class DuplicatesManager( object ):
class FileDuplicatesManager( object ):
def __init__( self, service_keys_to_dupe_statuses_to_counts ):
def __init__( self, media_group_king_hash, alternates_group_id, dupe_statuses_to_counts ):
self._service_keys_to_dupe_statuses_to_counts = service_keys_to_dupe_statuses_to_counts
self.media_group_king_hash = media_group_king_hash
self.alternates_group_id = alternates_group_id
self.dupe_statuses_to_count = dupe_statuses_to_counts
def Duplicate( self ):
service_keys_to_dupe_statuses_to_counts = collections.defaultdict( collections.Counter )
dupe_statuses_to_count = dict( self.dupe_statuses_to_count )
return DuplicatesManager( service_keys_to_dupe_statuses_to_counts )
return FileDuplicatesManager( self.media_group_king_hash, self.alternates_group_id, dupe_statuses_to_count )
def GetDupeStatusesToCounts( self, service_key ):
def GetDupeCount( self, dupe_type: int ):
return self._service_keys_to_dupe_statuses_to_counts[ service_key ]
if dupe_type not in self.dupe_statuses_to_count:
return 0
return self.dupe_statuses_to_count[ dupe_type ]
class FileInfoManager( object ):

2
hydrus/core/HydrusConstants.py
View File

@ -80,7 +80,7 @@ options = {}
# Misc
NETWORK_VERSION = 20
SOFTWARE_VERSION = 489
SOFTWARE_VERSION = 490
CLIENT_API_VERSION = 31
SERVER_THUMBNAIL_DIMENSIONS = ( 200, 200 )

3
hydrus/core/HydrusVideoHandling.py
View File

@ -1,7 +1,6 @@
import typing
import numpy
import numpy.typing
import os
import re
import struct
@ -1255,7 +1254,7 @@ class VideoRendererFFMPEG( object ):
def read_frame( self ) -> numpy.typing.ArrayLike:
def read_frame( self ):
if self.pos == self._num_frames: