RepoMirrors
/
hydrus
mirror of https://github.com/hydrusnetwork/hydrus.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Version 558

This commit is contained in:
Hydrus Network Developer 2024-01-10 15:27:29 -06:00
parent d40e2ecf12
commit 1ce383512d
No known key found for this signature in database
GPG Key ID: 76249F053212133C
27 changed files with 721 additions and 253 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

82
docs/changelog.md
View File

@ -7,6 +7,48 @@ title: Changelog
!!! note
This is the new changelog, only the most recent builds. For all versions, see the [old changelog](old_changelog.html).
## [Version 558](https://github.com/hydrusnetwork/hydrus/releases/tag/v558)
### user contributions
* thanks to a user, we now have rtf support! no word count yet, but it should be doable in future.
* thanks to a user, ctrl+p and ctrl+n now move the tag listbox selection up and down, in case the arrow keys aren't what you want. it also works on the tag autocomplete results from the text input
* added a link to 'Hydra Vista', https://github.com/konkrotte/hydravista, a macOS booru-like browser that talks to a hydrus client, to the main Client API help
### misc
* if you right-click on a selection of multiple tags, you can now hide them or their namespaces en masse
* if you right-click on a selection of multiple tags, you can now add or remove them from the favourites list en masse. if you select a mix of tags that are part-in, part-out of the list, you'll get both add and remove menu entries summarising what's going on. also, this command is now wrapped in a yes/no confirmation with full summary of what's being added/removed
* the 'favourites' "tag suggestions" section is renamed to 'most used'. this was often confused with the favourites that sit under a tag autocomplete, and these tags aren't really 'favourite' anyway, just most-used, so they are renamed
* if you have 'remove files from view when they are sent to the trash' set, then moving a file from one local file domain to another or removing one of multiple local file domains will no longer trigger a 'remove media'! sorry for the trouble, it was dumb logic on my part
* fixed the 'known urls' menu's url class section ('open all blahbooru urls' etc...) not appearing when right-clicking a single 'collection' thumbnail
* fixed the 'known urls' menu's open/copy specific urls not appearing when right-clicking any collection. it now shows the front 'display media's' urls
* if you change the darkmode in _options->colours_, the _help->darkmode_ menu item now updates correctly. just a side note: I hate much of this system and will eventually unify it with the style system
* fixed a bunch of 'number of x' tests at the database level when the operator is `≠`
### system:number of urls
* added `system:number of urls`! note this counts raw URLs at the moment--I just don't have fast database filtering of post urls vs file urls or url-classless urls or whatever. it does a raw count.
* `system:known urls` is now tucked with this new `system:number of urls` under a new stub predicate called `system:urls`
* a variety of 'system:number of words: has/no words' predicates now parse correctly when typed
* wrote some new system predicate parsing tests
### more cbz rules
* cbzs' non-image files must now have an appropriate extension like .txt, .nfo, or .xml
* the test regarding the count of non-image files (typically allowing up to 5 non-image files per directory) is more precise with regards to subdirectories, meaning a cbz with a single subdirectory and three non-image files now counts as a cbz
* every cbz must now have at least two image files that contain a number of some sort
### cleanup and boring stuff
* I split the github workflow build file into three, so the windows, linux, and macOS builds now all happen and upload in parallel. previously, the upload step was blocked on the slowest of the three, which was typically the macOS build by about ten minutes; now they all upload whenever they are ready. this will also help some future testing situations. the newly split scripts are a little unclean/inefficient, so there is also more work to do here
* I think I fixed the non-Windows executable permission bits for the various .sh and .command files in the base directory, which were lacking them, and I removed it from a couple dozen pngs across the docs and static directories, which somehow had them. let me know if I missed anything or messed anything up!
* if you click one of the static system predicate buttons that appear in the system pred edit UI, for instance 'system:has duration', this no longer gets promoted to the 'recent' predicates list the next time you open the panel
* some sytem predicate edit panels should stretch vertically a bit better
* some 'number of tags' queries should be a little faster
* the 'tag suggestions' options page has a bit of brushed up UI and some new explanation labels
* unified the various thumbnail generation error reporting for all the different filetypes. it should also print the file's hash, too, since most of these error contexts only have a temporary path to talk about at this stage, which isn't useful after the fact
## [Version 557](https://github.com/hydrusnetwork/hydrus/releases/tag/v557)
### misc
@ -415,43 +457,3 @@ title: Changelog
* the hydrus timedelta widget can now handle milliseconds
* misc code cleaning
* fixed a typo in the thumbnail 'select->local/not local' descriptions/tooltips
## [Version 548](https://github.com/hydrusnetwork/hydrus/releases/tag/v548)
### user contributions
* thanks to a user, krita files are now renderable! we've got the defaults set like psds for now, where the preview viewer will show 'open externally', but the media viewer tries to load the full thing. let's see how it goes, and as always, if you have one that doesn't work, please send it in! note that krita are now eligible for the similar files system, so I've queued them up to get entered into it
* thanks to a user, setting an IPFS 'nocopy' path including your home directory (~) should now expand correctly (issue #1320)
* thanks to a user, newly-IPFS-pinned files are properly aware of their multihashes now (previously you needed a client restart or media reload after a delay) (issue #1328)
* thanks to a user, the url and hdd downloaders now have 'stop/abort' buttons, which will stop current work and cancel the rest of the queue. I added a yes/no dialog where you can choose to skip or delete the remainder of the queue and a couple of bells and whistles like disabling the button when the current queue has no remaining work
### misc
* fixed an issue with successive drag and drop file exports that gave different files the same filename. previously, the successive files were being replaced with the first instance with the shared name (basically the original files were not being 'overwritten'), but it should be fixed now!
* various places that were sorting services pseudorandomly now do so alphabetically (the F9 new page selector was doing this with local file domains (the first buttons in 'file search'), if you had multiple set up. sorry if I mess with your muscle memory here, but things should be more reliable here going forward!)
* added a first version of an auto-update script, `auto_update_installer.bat`, to the main install directory. it will download the latest Windows exe installer using winget and install it to the current location. if you use the installer, you might want to experiment with it (make a backup first!) as an easy hands-free update solution. let me know how it goes, and if there are no problems in a couple of weeks, I'll add it to the help
* added some more mpv error handling. if the mainloop behind your mpv window halts (which happens on various internal problems), we now detect it and more gracefully disable the viewer and its commands (previously it would escalate to error popups and try to keep working)
* fixed an issue in the newer 'missing file storage recovery' code if there is more than one base location missing
### thumbnail shortcuts
* I converted all the old hardcoded thumbnail keyboard shortcuts (thumbnail focus movement, open-media-viewer, and select-files) to the newer user-editable system under _file->shortcuts_, under a new set called 'thumbnails'. there are some new file-filters too, so you can set up 'select inbox' and similar beyond the default ctrl+a to 'select all' and escape to 'select none'
* I don't expect many people will want to even touch the giganto list of (shift+)(numpad)left/right/up/down/page up/page down/home/end selection combinations, but if you want to, you can!
* the thumbnails set also now allows 'launch the archive/delete filter', which had an odd home in 'media' before. new users now start with F12 set up in 'thumbnails', not 'media'
* I removed the jank semi-secret 'ctrl+space' hardcoded 'deselect current focused thumbnail' shortcut. that tech will probably return when I figure out more sensible logic and user settings around shift+ and ctrl+ behaviour
* this cleanup reduces three different shortcut handling routines down to one, and it particularly clears the last place where I was using ancient grandfathered wx-based 'accelerator table' tech. it should be easier to update the thumbnail shortcuts in future, and I hope to plug the mouse into it also, so you can edit middle-click to launch media etc..
### client api
* after much discussion and personal vacillating, I have decided to include the `version` and `hydrus_version` in every JSON Client API response. CBOR responses are not affected. if you need to hook into these numbers for a completely stateless interface, it is now super convenient. I'm not delighted with the spamminess of this, but it is just a handful of characters and it adds value for several situations, so I'm willing to try it out
* updated the documentation and unit tests regarding this
* the client api version is now 54
### boring stuff
* file filter objects are now serialisable
* application commands can now hold serialisable objects in their 'simple data' slot
* I made a new 'slightly more than simple' application command to hold a 'thumbnail move' that has both a direction and a selection status. I expect it will be expanded in future to handle ctrl+ selection and other logic preferences
* I made a new application command to hold the file filter. I just pre-populate the UI with a dropdown with commond choices for now, but in future it could hold a customisable file filter, once, ha ha, I have some UI to actually edit one!
* cleaned up various shortcut code
* misc linting cleanup

1
docs/client_api.md
View File

@ -30,6 +30,7 @@ Once the API is running, go to its entry in _services->review services_. Each ex
* [Hydrus Web](https://github.com/floogulinc/hydrus-web): a web client for hydrus (allows phone browsing of hydrus)
* [Hyshare](https://github.com/floogulinc/hyshare): a way to share small galleries with friends--a replacement for the old 'local booru' system
* [Hydra Vista](https://github.com/konkrotte/hydravista): a macOS client for hydrus
* [LoliSnatcher](https://github.com/NO-ob/LoliSnatcher_Droid): a booru client for Android that can talk to hydrus
* [Anime Boxes](https://www.animebox.es/): a booru browser, now supports adding your client as a Hydrus Server
* [FlipFlip](https://ififfy.github.io/flipflip/#/): an advanced slideshow interface, now supports hydrus as a source

35
docs/old_changelog.html
View File

@ -34,6 +34,41 @@
<div class="content">
<h1 id="changelog"><a href="#changelog">changelog</a></h1>
<ul>
<li>
<h2 id="version_558"><a href="#version_558">version 558</a></h2>
<ul>
<li><h3>user contributions</h3></li>
<li>thanks to a user, we now have rtf support! no word count yet, but it should be doable in future.</li>
<li>thanks to a user, ctrl+p and ctrl+n now move the tag listbox selection up and down, in case the arrow keys aren't what you want. it also works on the tag autocomplete results from the text input</li>
<li>added a link to 'Hydra Vista', https://github.com/konkrotte/hydravista, a macOS booru-like browser that talks to a hydrus client, to the main Client API help</li>
<li><h3>misc</h3></li>
<li>if you right-click on a selection of multiple tags, you can now hide them or their namespaces en masse</li>
<li>if you right-click on a selection of multiple tags, you can now add or remove them from the favourites list en masse. if you select a mix of tags that are part-in, part-out of the list, you'll get both add and remove menu entries summarising what's going on. also, this command is now wrapped in a yes/no confirmation with full summary of what's being added/removed</li>
<li>the 'favourites' "tag suggestions" section is renamed to 'most used'. this was often confused with the favourites that sit under a tag autocomplete, and these tags aren't really 'favourite' anyway, just most-used, so they are renamed</li>
<li>if you have 'remove files from view when they are sent to the trash' set, then moving a file from one local file domain to another or removing one of multiple local file domains will no longer trigger a 'remove media'! sorry for the trouble, it was dumb logic on my part</li>
<li>fixed the 'known urls' menu's url class section ('open all blahbooru urls' etc...) not appearing when right-clicking a single 'collection' thumbnail</li>
<li>fixed the 'known urls' menu's open/copy specific urls not appearing when right-clicking any collection. it now shows the front 'display media's' urls</li>
<li>if you change the darkmode in _options->colours_, the _help->darkmode_ menu item now updates correctly. just a side note: I hate much of this system and will eventually unify it with the style system</li>
<li>fixed a bunch of 'number of x' tests at the database level when the operator is `≠`</li>
<li><h3>system:number of urls</h3></li>
<li>added `system:number of urls`! note this counts raw URLs at the moment--I just don't have fast database filtering of post urls vs file urls or url-classless urls or whatever. it does a raw count.</li>
<li>`system:known urls` is now tucked with this new `system:number of urls` under a new stub predicate called `system:urls`</li>
<li>a variety of 'system:number of words: has/no words' predicates now parse correctly when typed</li>
<li>wrote some new system predicate parsing tests</li>
<li><h3>more cbz rules</h3></li>
<li>cbzs' non-image files must now have an appropriate extension like .txt, .nfo, or .xml</li>
<li>the test regarding the count of non-image files (typically allowing up to 5 non-image files per directory) is more precise with regards to subdirectories, meaning a cbz with a single subdirectory and three non-image files now counts as a cbz</li>
<li>every cbz must now have at least two image files that contain a number of some sort</li>
<li><h3>cleanup and boring stuff</h3></li>
<li>I split the github workflow build file into three, so the windows, linux, and macOS builds now all happen and upload in parallel. previously, the upload step was blocked on the slowest of the three, which was typically the macOS build by about ten minutes; now they all upload whenever they are ready. this will also help some future testing situations. the newly split scripts are a little unclean/inefficient, so there is also more work to do here</li>
<li>I think I fixed the non-Windows executable permission bits for the various .sh and .command files in the base directory, which were lacking them, and I removed it from a couple dozen pngs across the docs and static directories, which somehow had them. let me know if I missed anything or messed anything up!</li>
<li>if you click one of the static system predicate buttons that appear in the system pred edit UI, for instance 'system:has duration', this no longer gets promoted to the 'recent' predicates list the next time you open the panel</li>
<li>some sytem predicate edit panels should stretch vertically a bit better</li>
<li>some 'number of tags' queries should be a little faster</li>
<li>the 'tag suggestions' options page has a bit of brushed up UI and some new explanation labels</li>
<li>unified the various thumbnail generation error reporting for all the different filetypes. it should also print the file's hash, too, since most of these error contexts only have a temporary path to talk about at this stage, which isn't useful after the fact</li>
</ul>
</li>
<li>
<h2 id="version_557"><a href="#version_557">version 557</a></h2>
<ul>

2
hydrus/client/db/ClientDB.py
View File

@ -3274,7 +3274,6 @@ class DB( HydrusDB.HydrusDB ):
blank_pred_types = {
ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_TAGS,
ClientSearch.PREDICATE_TYPE_SYSTEM_LIMIT,
ClientSearch.PREDICATE_TYPE_SYSTEM_KNOWN_URLS,
ClientSearch.PREDICATE_TYPE_SYSTEM_HASH,
ClientSearch.PREDICATE_TYPE_SYSTEM_FILE_SERVICE,
ClientSearch.PREDICATE_TYPE_SYSTEM_FILE_RELATIONSHIPS,
@ -3435,6 +3434,7 @@ class DB( HydrusDB.HydrusDB ):
ClientSearch.PREDICATE_TYPE_SYSTEM_FILE_PROPERTIES,
ClientSearch.PREDICATE_TYPE_SYSTEM_NOTES,
ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_WORDS,
ClientSearch.PREDICATE_TYPE_SYSTEM_URLS,
ClientSearch.PREDICATE_TYPE_SYSTEM_MIME,
ClientSearch.PREDICATE_TYPE_SYSTEM_SIMILAR_TO
] )

41
hydrus/client/db/ClientDBFilesSearch.py
View File

@ -2174,6 +2174,18 @@ class ClientDBFilesQuery( ClientDBModule.ClientDBModule ):
#
num_urls_tests = system_predicates.GetNumURLsNumberTests()
if len( num_urls_tests ) > 0:
with self._MakeTemporaryIntegerTable( query_hash_ids, 'hash_id' ) as temp_table_name:
url_hash_ids = self.modules_url_map.GetHashIdsFromCountTests( num_urls_tests, query_hash_ids, temp_table_name )
query_hash_ids = intersection_update_qhi( query_hash_ids, url_hash_ids )
if 'known_url_rules' in simple_preds:
for ( operator, rule_type, rule ) in simple_preds[ 'known_url_rules' ]:
@ -2216,26 +2228,23 @@ class ClientDBFilesQuery( ClientDBModule.ClientDBModule ):
namespace_wildcard = '*'
is_zero = True in ( number_test.IsZero() for number_test in number_tests )
is_anything_but_zero = True in ( number_test.IsAnythingButZero() for number_test in number_tests )
specific_number_tests = [ number_test for number_test in number_tests if not ( number_test.IsZero() or number_test.IsAnythingButZero() ) ]
lambdas = [ number_test.GetLambda() for number_test in specific_number_tests ]
megalambda = ClientSearch.NumberTest.STATICCreateMegaLambda( specific_number_tests )
megalambda = lambda x: False not in ( lamb( x ) for lamb in lambdas )
is_zero = True in ( number_test.IsZero() for number_test in number_tests )
is_anything_but_zero = True in ( number_test.IsAnythingButZero() for number_test in number_tests )
wants_zero = True in ( number_test.WantsZero() for number_test in number_tests )
with self._MakeTemporaryIntegerTable( query_hash_ids, 'hash_id' ) as temp_table_name:
nonzero_tag_query_hash_ids = set()
if is_zero or is_anything_but_zero or wants_zero:
self._AnalyzeTempTable( temp_table_name )
nonzero_tag_query_hash_ids = set()
nonzero_tag_query_hash_ids_populated = False
if is_zero or is_anything_but_zero:
with self._MakeTemporaryIntegerTable( query_hash_ids, 'hash_id' ) as temp_table_name:
self._AnalyzeTempTable( temp_table_name )
nonzero_tag_query_hash_ids = self.modules_files_search_tags.GetHashIdsThatHaveTagsComplexLocation( ClientTags.TAG_DISPLAY_DISPLAY_ACTUAL, location_context, tag_context, hash_ids_table_name = temp_table_name, namespace_wildcard = namespace_wildcard, job_status = job_status )
nonzero_tag_query_hash_ids_populated = True
if is_zero:
@ -2257,11 +2266,6 @@ class ClientDBFilesQuery( ClientDBModule.ClientDBModule ):
if megalambda( 0 ): # files with zero count are needed
if not nonzero_tag_query_hash_ids_populated:
nonzero_tag_query_hash_ids = { hash_id for ( hash_id, count ) in hash_id_tag_counts }
zero_hash_ids = query_hash_ids.difference( nonzero_tag_query_hash_ids )
good_tag_count_hash_ids.update( zero_hash_ids )
@ -2270,7 +2274,6 @@ class ClientDBFilesQuery( ClientDBModule.ClientDBModule ):
query_hash_ids = intersection_update_qhi( query_hash_ids, good_tag_count_hash_ids )
if job_status.IsCancelled():

61
hydrus/client/db/ClientDBURLMap.py
View File

@ -4,10 +4,10 @@ import typing
from hydrus.core import HydrusConstants as HC
from hydrus.core import HydrusData
from hydrus.core import HydrusTime
from hydrus.client.db import ClientDBMaster
from hydrus.client.db import ClientDBModule
from hydrus.client.search import ClientSearch
class ClientDBURLMap( ClientDBModule.ClientDBModule ):
@ -57,6 +57,65 @@ class ClientDBURLMap( ClientDBModule.ClientDBModule ):
return hash_ids
def GetHashIdsFromCountTests( self, num_urls_tests: typing.List[ ClientSearch.NumberTest ], hash_ids: typing.Collection[ int ], hash_ids_table_name: str ):
# we'll have to natural join 'urls' or 'urls-class-map-cache' or whatever when we add a proper filter to this guy
table_join = 'url_map'
if len( hash_ids ) < 50000:
table_join += ' NATURAL JOIN {}'.format( hash_ids_table_name )
#
result_hash_ids = set( hash_ids )
specific_num_urls_tests = [ number_test for number_test in num_urls_tests if not ( number_test.IsZero() or number_test.IsAnythingButZero() ) ]
megalambda = ClientSearch.NumberTest.STATICCreateMegaLambda( specific_num_urls_tests )
is_zero = True in ( number_test.IsZero() for number_test in num_urls_tests )
is_anything_but_zero = True in ( number_test.IsAnythingButZero() for number_test in num_urls_tests )
wants_zero = True in ( number_test.WantsZero() for number_test in num_urls_tests )
if is_zero or is_anything_but_zero or wants_zero:
select = f'SELECT DISTINCT hash_id FROM {table_join};'
nonzero_url_query_hash_ids = self._STS( self._Execute( select ) )
if is_zero:
result_hash_ids.difference_update( nonzero_url_query_hash_ids )
if is_anything_but_zero:
result_hash_ids.intersection_update( nonzero_url_query_hash_ids )
if len( specific_num_urls_tests ) > 0:
select = f'SELECT hash_id, COUNT( url_id ) FROM {table_join} GROUP BY hash_id;'
good_url_count_hash_ids = { hash_id for ( hash_id, count ) in self._Execute( select ) if megalambda( count ) }
if wants_zero:
zero_hash_ids = result_hash_ids.difference( nonzero_url_query_hash_ids )
good_url_count_hash_ids.update( zero_hash_ids )
result_hash_ids.intersection_update( good_url_count_hash_ids )
return result_hash_ids
def GetHashIdsFromURLRule( self, rule_type, rule, hash_ids = None, hash_ids_table_name = None ):
if rule_type == 'exact_match':

4
hydrus/client/gui/ClientGUI.py
View File

@ -3320,7 +3320,7 @@ class FrameGUI( CAC.ApplicationCommandProcessorMixin, ClientGUITopLevelWindows.M
currently_darkmode = self._new_options.GetString( 'current_colourset' ) == 'darkmode'
ClientGUIMenus.AppendMenuCheckItem( menu, 'darkmode', 'Set the \'darkmode\' colourset on and off.', currently_darkmode, self.FlipDarkmode )
self._menu_item_help_darkmode = ClientGUIMenus.AppendMenuCheckItem( menu, 'darkmode', 'Set the \'darkmode\' colourset on and off.', currently_darkmode, self.FlipDarkmode )
check_manager = ClientGUICommon.CheckboxManagerOptions( 'advanced_mode' )
@ -4374,6 +4374,8 @@ class FrameGUI( CAC.ApplicationCommandProcessorMixin, ClientGUITopLevelWindows.M
self._controller.pub( 'notify_new_colourset' )
self._controller.pub( 'notify_new_favourite_tags' )
self._menu_item_help_darkmode.setChecked( HG.client_controller.new_options.GetString( 'current_colourset' ) == 'darkmode' )
self._UpdateSystemTrayIcon()

72
hydrus/client/gui/ClientGUIMediaMenus.py
View File

@ -348,6 +348,11 @@ def AddKnownURLsViewCopyMenu( win, menu, focus_media, selected_media = None ):
# figure out which urls this focused file has
if focus_media.IsCollection():
focus_media = focus_media.GetDisplayMedia()
focus_urls = focus_media.GetLocationsManager().GetURLs()
focus_matched_labels_and_urls = []
@ -392,50 +397,53 @@ def AddKnownURLsViewCopyMenu( win, menu, focus_media, selected_media = None ):
selected_media_url_classes = set()
multiple_or_unmatching_selection_url_classes = False
if selected_media is not None and len( selected_media ) > 1:
if selected_media is not None:
selected_media = ClientMedia.FlattenMedia( selected_media )
SAMPLE_SIZE = 256
if len( selected_media ) > SAMPLE_SIZE:
if len( selected_media ) > 1:
selected_media_sample = random.sample( selected_media, SAMPLE_SIZE )
SAMPLE_SIZE = 256
else:
selected_media_sample = selected_media
for media in selected_media_sample:
media_urls = media.GetLocationsManager().GetURLs()
for url in media_urls:
if len( selected_media ) > SAMPLE_SIZE:
try:
url_class = HG.client_controller.network_engine.domain_manager.GetURLClass( url )
except HydrusExceptions.URLClassException:
continue
selected_media_sample = random.sample( selected_media, SAMPLE_SIZE )
if url_class is None:
else:
selected_media_sample = selected_media
for media in selected_media_sample:
media_urls = media.GetLocationsManager().GetURLs()
for url in media_urls:
multiple_or_unmatching_selection_url_classes = True
try:
url_class = HG.client_controller.network_engine.domain_manager.GetURLClass( url )
except HydrusExceptions.URLClassException:
continue
else:
selected_media_url_classes.add( url_class )
if url_class is None:
multiple_or_unmatching_selection_url_classes = True
else:
selected_media_url_classes.add( url_class )
if len( selected_media_url_classes ) > 1:
multiple_or_unmatching_selection_url_classes = True
if len( selected_media_url_classes ) > 1:
multiple_or_unmatching_selection_url_classes = True

37
hydrus/client/gui/ClientGUIScrolledPanelsManagement.py
View File

@ -4383,7 +4383,9 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
for item in [ 'favourites', 'related', 'file_lookup_scripts', 'recent' ]:
self._default_suggested_tags_notebook_page.addItem( item, item )
label = 'most used' if item == 'favourites' else item
self._default_suggested_tags_notebook_page.addItem( label, item )
suggest_tags_panel_notebook = QW.QTabWidget( suggested_tags_panel )
@ -4513,7 +4515,18 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
panel_vbox = QP.VBoxLayout()
QP.AddToLayout( panel_vbox, self._suggested_favourites_services, CC.FLAGS_EXPAND_PERPENDICULAR )
st = ClientGUICommon.BetterStaticText( suggested_tags_favourites_panel, 'Add your most used tags for each particular service here, and then you can just double-click to add, rather than typing every time.' )
st.setWordWrap( True )
QP.AddToLayout( panel_vbox, st, CC.FLAGS_EXPAND_PERPENDICULAR )
rows = []
rows.append( ( 'Tag service: ', self._suggested_favourites_services ) )
gridbox = ClientGUICommon.WrapInGrid( suggested_tags_related_panel, rows )
QP.AddToLayout( panel_vbox, gridbox, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
QP.AddToLayout( panel_vbox, self._suggested_favourites, CC.FLAGS_EXPAND_BOTH_WAYS )
QP.AddToLayout( panel_vbox, self._suggested_favourites_input, CC.FLAGS_EXPAND_PERPENDICULAR )
@ -4533,13 +4546,15 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
gridbox = ClientGUICommon.WrapInGrid( suggested_tags_related_panel, rows )
desc = 'This will search the database for tags statistically related to what your files already have.'
search_tag_slices_weight_box.Add( search_tag_slices_weight_panel, CC.FLAGS_EXPAND_BOTH_WAYS )
result_tag_slices_weight_box.Add( result_tag_slices_weight_panel, CC.FLAGS_EXPAND_BOTH_WAYS )
QP.AddToLayout( panel_vbox, ClientGUICommon.BetterStaticText( suggested_tags_related_panel, desc ), CC.FLAGS_EXPAND_PERPENDICULAR )
desc = 'This will search the database for tags statistically related to what your files already have. It only searches within the specific service atm. The score weights are advanced, so only change them if you know what is going on!'
st = ClientGUICommon.BetterStaticText( suggested_tags_related_panel, desc )
st.setWordWrap( True )
QP.AddToLayout( panel_vbox, st, CC.FLAGS_EXPAND_PERPENDICULAR )
QP.AddToLayout( panel_vbox, gridbox, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
QP.AddToLayout( panel_vbox, search_tag_slices_weight_box, CC.FLAGS_EXPAND_BOTH_WAYS )
QP.AddToLayout( panel_vbox, result_tag_slices_weight_box, CC.FLAGS_EXPAND_BOTH_WAYS )
@ -4557,6 +4572,11 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
gridbox = ClientGUICommon.WrapInGrid( suggested_tags_file_lookup_script_panel, rows )
desc = 'This is an increasingly defunct system, do not expect miracles!'
st = ClientGUICommon.BetterStaticText( suggested_tags_related_panel, desc )
st.setWordWrap( True )
QP.AddToLayout( panel_vbox, st, CC.FLAGS_EXPAND_PERPENDICULAR )
QP.AddToLayout( panel_vbox, gridbox, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
suggested_tags_file_lookup_script_panel.setLayout( panel_vbox )
@ -4565,6 +4585,11 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
panel_vbox = QP.VBoxLayout()
desc = 'This simply saves the last n tags you have added for each service.'
st = ClientGUICommon.BetterStaticText( suggested_tags_related_panel, desc )
st.setWordWrap( True )
QP.AddToLayout( panel_vbox, st, CC.FLAGS_EXPAND_PERPENDICULAR )
QP.AddToLayout( panel_vbox, self._num_recent_tags, CC.FLAGS_EXPAND_PERPENDICULAR )
panel_vbox.addStretch( 1 )
@ -4573,7 +4598,7 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
#
suggest_tags_panel_notebook.addTab( suggested_tags_favourites_panel, 'favourites' )
suggest_tags_panel_notebook.addTab( suggested_tags_favourites_panel, 'most used' )
suggest_tags_panel_notebook.addTab( suggested_tags_related_panel, 'related' )
suggest_tags_panel_notebook.addTab( suggested_tags_file_lookup_script_panel, 'file lookup scripts' )
suggest_tags_panel_notebook.addTab( suggested_tags_recent_panel, 'recent' )

3
hydrus/client/gui/ClientGUITagSuggestions.py
View File

@ -821,7 +821,7 @@ class SuggestedTagsPanel( QW.QWidget ):
self._favourite_tags.mouseActivationOccurred.connect( self.mouseActivationOccurred )
panels.append( ( 'favourites', self._favourite_tags ) )
panels.append( ( 'most used', self._favourite_tags ) )
self._related_tags = None
@ -870,6 +870,7 @@ class SuggestedTagsPanel( QW.QWidget ):
name_to_page_dict = {
'favourites' : self._favourite_tags,
'most used' : self._favourite_tags,
'related' : self._related_tags,
'file_lookup_scripts' : self._file_lookup_script_tags,
'recent' : self._recent_tags

221
hydrus/client/gui/lists/ClientGUIListBoxes.py
View File

@ -2536,59 +2536,43 @@ class ListBoxTags( ListBox ):
if command in ( 'hide', 'hide_namespace' ):
if len( tags ) == 1:
if command == 'hide':
( tag, ) = tags
message = f'Hide{HydrusData.ConvertManyStringsToNiceInsertableHumanSummary( tags )}from here?'
if command == 'hide':
from hydrus.client.gui import ClientGUIDialogsQuick
result = ClientGUIDialogsQuick.GetYesNo( self, message )
if result != QW.QDialog.Accepted:
message = 'Hide "{}" from here?'.format( tag )
from hydrus.client.gui import ClientGUIDialogsQuick
result = ClientGUIDialogsQuick.GetYesNo( self, message )
if result != QW.QDialog.Accepted:
return
HG.client_controller.tag_display_manager.HideTag( self._tag_display_type, CC.COMBINED_TAG_SERVICE_KEY, tag )
elif command == 'hide_namespace':
( namespace, subtag ) = HydrusTags.SplitTag( tag )
if namespace == '':
insert = 'unnamespaced'
else:
insert = '"{}"'.format( namespace )
message = 'Hide {} tags from here?'.format( insert )
from hydrus.client.gui import ClientGUIDialogsQuick
result = ClientGUIDialogsQuick.GetYesNo( self, message )
if result != QW.QDialog.Accepted:
return
if namespace != '':
namespace += ':'
HG.client_controller.tag_display_manager.HideTag( self._tag_display_type, CC.COMBINED_TAG_SERVICE_KEY, namespace )
return
HG.client_controller.pub( 'notify_new_tag_display_rules' )
HG.client_controller.tag_display_manager.HideTags( self._tag_display_type, CC.COMBINED_TAG_SERVICE_KEY, tags )
elif command == 'hide_namespace':
namespaces = { namespace for ( namespace, subtag ) in ( HydrusTags.SplitTag( tag ) for tag in tags ) }
nice_namespaces = [ ClientTags.RenderNamespaceForUser( namespace ) for namespace in namespaces ]
message = f'Hide{HydrusData.ConvertManyStringsToNiceInsertableHumanSummary( nice_namespaces )}tags from here?'
from hydrus.client.gui import ClientGUIDialogsQuick
result = ClientGUIDialogsQuick.GetYesNo( self, message )
if result != QW.QDialog.Accepted:
return
tag_slices = [ namespace if namespace == '' else namespace + ':' for namespace in namespaces ]
HG.client_controller.tag_display_manager.HideTags( self._tag_display_type, CC.COMBINED_TAG_SERVICE_KEY, tag_slices )
HG.client_controller.pub( 'notify_new_tag_display_rules' )
else:
@ -3255,61 +3239,144 @@ class ListBoxTags( ListBox ):
if len( selected_actual_tags ) == 1:
( selected_tag, ) = selected_actual_tags
if len( selected_actual_tags ) > 0:
if self._tag_display_type in ( ClientTags.TAG_DISPLAY_SINGLE_MEDIA, ClientTags.TAG_DISPLAY_SELECTION_LIST ):
ClientGUIMenus.AppendSeparator( menu )
( namespace, subtag ) = HydrusTags.SplitTag( selected_tag )
namespaces = set()
for selected_actual_tag in selected_actual_tags:
( namespace, subtag ) = HydrusTags.SplitTag( selected_actual_tag )
namespaces.add( namespace )
if len( namespaces ) == 1:
namespace = list( namespaces )[0]
namespace_label = f'"{ClientTags.RenderNamespaceForUser( namespace )}" tags from here'
else:
namespace_label = f'{HydrusData.ToHumanInt( len( namespaces ) )} selected namespaces from here'
if len( selected_actual_tags ) == 1:
actual_tag = list( selected_actual_tags )[0]
actual_tag_label = f'"{actual_tag}" from here'
else:
actual_tag_label = f'{HydrusData.ToHumanInt( len( selected_actual_tags ) )} selected tags from here'
hide_menu = ClientGUIMenus.GenerateMenu( menu )
ClientGUIMenus.AppendMenuItem( hide_menu, '"{}" tags from here'.format( ClientTags.RenderNamespaceForUser( namespace ) ), 'Hide this namespace from view in future.', self._ProcessMenuTagEvent, 'hide_namespace' )
ClientGUIMenus.AppendMenuItem( hide_menu, '"{}" from here'.format( selected_tag ), 'Hide this tag from view in future.', self._ProcessMenuTagEvent, 'hide' )
ClientGUIMenus.AppendMenuItem( hide_menu, namespace_label, 'Hide these namespaces from view in future.', self._ProcessMenuTagEvent, 'hide_namespace' )
ClientGUIMenus.AppendMenuItem( hide_menu, actual_tag_label, 'Hide these tags from view in future.', self._ProcessMenuTagEvent, 'hide' )
ClientGUIMenus.AppendMenu( menu, hide_menu, 'hide' )
def set_favourite_tags( tag ):
#
def add_favourite_tags( tags ):
message = f'Add{HydrusData.ConvertManyStringsToNiceInsertableHumanSummary( tags )}to the favourites list?'
from hydrus.client.gui import ClientGUIDialogsQuick
result = ClientGUIDialogsQuick.GetYesNo( self, message )
if result != QW.QDialog.Accepted:
favourite_tags = list( HG.client_controller.new_options.GetStringList( 'favourite_tags' ) )
if selected_tag in favourite_tags:
favourite_tags.remove( tag )
else:
favourite_tags.append( tag )
HG.client_controller.new_options.SetStringList( 'favourite_tags', favourite_tags )
HG.client_controller.pub( 'notify_new_favourite_tags' )
return
favourite_tags = list( HG.client_controller.new_options.GetStringList( 'favourite_tags' ) )
favourite_tags = set( HG.client_controller.new_options.GetStringList( 'favourite_tags' ) )
if selected_tag in favourite_tags:
favourite_tags.update( tags )
HG.client_controller.new_options.SetStringList( 'favourite_tags', list( favourite_tags ) )
HG.client_controller.pub( 'notify_new_favourite_tags' )
def remove_favourite_tags( tags ):
message = f'Remove{HydrusData.ConvertManyStringsToNiceInsertableHumanSummary( tags )}from the favourites list?'
from hydrus.client.gui import ClientGUIDialogsQuick
result = ClientGUIDialogsQuick.GetYesNo( self, message )
if result != QW.QDialog.Accepted:
label = 'remove "{}" from favourites'.format( selected_tag )
description = 'Remove this tag from your favourites'
return
favourite_tags = set( HG.client_controller.new_options.GetStringList( 'favourite_tags' ) )
favourite_tags.difference_update( tags )
HG.client_controller.new_options.SetStringList( 'favourite_tags', list( favourite_tags ) )
HG.client_controller.pub( 'notify_new_favourite_tags' )
favourite_tags = list( HG.client_controller.new_options.GetStringList( 'favourite_tags' ) )
to_add = set( selected_actual_tags ).difference( favourite_tags )
to_remove = set( selected_actual_tags ).intersection( favourite_tags )
favourites_menu = ClientGUIMenus.GenerateMenu( menu )
if len( to_add ) > 0:
if len( to_add ) == 1:
tag = list( to_add )[0]
label = f'Add "{tag}" to favourites'
else:
label = 'add "{}" to favourites'.format( selected_tag )
description = 'Add this tag from your favourites'
label = f'Add {HydrusData.ToHumanInt( len( to_add ) )} selected tags to favourites'
favourites_menu = ClientGUIMenus.GenerateMenu( menu )
description = 'Add these tags to the favourites list.'
ClientGUIMenus.AppendMenuItem( favourites_menu, label, description, set_favourite_tags, selected_tag )
ClientGUIMenus.AppendMenuItem( favourites_menu, label, description, add_favourite_tags, to_add )
m = ClientGUIMenus.AppendMenu( menu, favourites_menu, 'favourites' )
if len( to_remove ) > 0:
if len( to_remove ) == 1:
tag = list( to_remove )[0]
label = f'Remove "{tag}" from favourites'
else:
label = f'Remove {HydrusData.ToHumanInt( len( to_remove ) )} selected tags from favourites'
description = 'Add these tags to the favourites list.'
ClientGUIMenus.AppendMenuItem( favourites_menu, label, description, remove_favourite_tags, to_remove )
ClientGUIMenus.AppendMenu( menu, favourites_menu, 'favourites' )
#
self.AddAdditionalMenuItems( menu )

51
hydrus/client/gui/search/ClientGUIPredicatesSingle.py
View File

@ -1936,6 +1936,56 @@ class PanelPredicateSystemNumNotes( PanelPredicateSystemSingle ):
return predicates
class PanelPredicateSystemNumURLs( PanelPredicateSystemSingle ):
def __init__( self, parent, predicate ):
PanelPredicateSystemSingle.__init__( self, parent )
self._sign = QP.RadioBox( self, choices=['<','=',HC.UNICODE_NOT_EQUAL,'>'] )
self._num_urls = ClientGUICommon.BetterSpinBox( self, max=1000000, width = 60 )
#
predicate = self._GetPredicateToInitialisePanelWith( predicate )
( sign, num_urls ) = predicate.GetValue()
self._sign.SetStringSelection( sign )
self._num_urls.setValue( num_urls )
#
hbox = QP.HBoxLayout()
QP.AddToLayout( hbox, ClientGUICommon.BetterStaticText( self,'system:number of urls' ), CC.FLAGS_CENTER_PERPENDICULAR )
QP.AddToLayout( hbox, self._sign, CC.FLAGS_CENTER_PERPENDICULAR )
QP.AddToLayout( hbox, self._num_urls, CC.FLAGS_CENTER_PERPENDICULAR )
hbox.addStretch( 1 )
self.setLayout( hbox )
def GetDefaultPredicate( self ):
sign = '>'
num_urls = 0
return ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_URLS, ( sign, num_urls ) )
def GetPredicates( self ):
predicates = ( ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_URLS, ( self._sign.GetStringSelection(), self._num_urls.value() ) ), )
return predicates
class PanelPredicateSystemNumWords( PanelPredicateSystemSingle ):
def __init__( self, parent, predicate ):
@ -1984,6 +2034,7 @@ class PanelPredicateSystemNumWords( PanelPredicateSystemSingle ):
return predicates
class PanelPredicateSystemRatio( PanelPredicateSystemSingle ):
def __init__( self, parent, predicate ):

57
hydrus/client/gui/search/ClientGUISearch.py
View File

@ -41,6 +41,7 @@ FLESH_OUT_SYSTEM_PRED_TYPES = {
ClientSearch.PREDICATE_TYPE_SYSTEM_HAS_ICC_PROFILE,
ClientSearch.PREDICATE_TYPE_SYSTEM_HAS_FORCED_FILETYPE,
ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_WORDS,
ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_URLS,
ClientSearch.PREDICATE_TYPE_SYSTEM_MIME,
ClientSearch.PREDICATE_TYPE_SYSTEM_RATING,
ClientSearch.PREDICATE_TYPE_SYSTEM_SIMILAR_TO,
@ -48,6 +49,7 @@ FLESH_OUT_SYSTEM_PRED_TYPES = {
ClientSearch.PREDICATE_TYPE_SYSTEM_SIMILAR_TO_FILES,
ClientSearch.PREDICATE_TYPE_SYSTEM_FILE_SERVICE,
ClientSearch.PREDICATE_TYPE_SYSTEM_TAG_AS_NUMBER,
ClientSearch.PREDICATE_TYPE_SYSTEM_URLS,
ClientSearch.PREDICATE_TYPE_SYSTEM_FILE_RELATIONSHIPS,
ClientSearch.PREDICATE_TYPE_SYSTEM_NOTES,
ClientSearch.PREDICATE_TYPE_SYSTEM_TIME,
@ -321,6 +323,10 @@ class EditPredicatesPanel( ClientGUIScrolledPanels.EditPanel ):
self._editable_pred_panels.append( ClientGUIPredicatesSingle.PanelPredicateSystemKnownURLsURLClass( self, predicate ) )
elif predicate_type == ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_URLS:
self._editable_pred_panels.append( ClientGUIPredicatesSingle.PanelPredicateSystemNumURLs( self, predicate ) )
elif predicate_type == ClientSearch.PREDICATE_TYPE_SYSTEM_HASH:
self._editable_pred_panels.append( ClientGUIPredicatesSingle.PanelPredicateSystemHash( self, predicate ) )
@ -569,6 +575,34 @@ class FleshOutPredicatePanel( ClientGUIScrolledPanels.EditPanel ):
editable_pred_panels.append( self._PredOKPanel( self, ClientGUIPredicatesSingle.PanelPredicateSystemKnownURLsRegex, predicate ) )
editable_pred_panels.append( self._PredOKPanel( self, ClientGUIPredicatesSingle.PanelPredicateSystemKnownURLsURLClass, predicate ) )
elif predicate_type == ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_URLS:
editable_pred_panels.append( self._PredOKPanel( self, ClientGUIPredicatesSingle.PanelPredicateSystemNumURLs, predicate ) )
elif predicate_type == ClientSearch.PREDICATE_TYPE_SYSTEM_URLS:
label = 'Note that "number of urls" counts all URLs, regardless of how important.'
recent_predicate_types = [ ClientSearch.PREDICATE_TYPE_SYSTEM_KNOWN_URLS ]
editable_pred_panels.append( self._PredOKPanel( self, ClientGUIPredicatesSingle.PanelPredicateSystemKnownURLsExactURL, predicate ) )
editable_pred_panels.append( self._PredOKPanel( self, ClientGUIPredicatesSingle.PanelPredicateSystemKnownURLsDomain, predicate ) )
editable_pred_panels.append( self._PredOKPanel( self, ClientGUIPredicatesSingle.PanelPredicateSystemKnownURLsRegex, predicate ) )
editable_pred_panels.append( self._PredOKPanel( self, ClientGUIPredicatesSingle.PanelPredicateSystemKnownURLsURLClass, predicate ) )
pages.append( ( 'known urls', recent_predicate_types, static_pred_buttons, editable_pred_panels ) )
page_name = 'number of urls'
recent_predicate_types = [ ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_URLS ]
static_pred_buttons = []
editable_pred_panels = []
static_pred_buttons.append( ClientGUIPredicatesSingle.StaticSystemPredicateButton( self, ( ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_URLS, ( '>', 0 ) ), ), show_remove_button = False ) )
static_pred_buttons.append( ClientGUIPredicatesSingle.StaticSystemPredicateButton( self, ( ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_URLS, ( '=', 0 ) ), ), show_remove_button = False ) )
editable_pred_panels.append( self._PredOKPanel( self, ClientGUIPredicatesSingle.PanelPredicateSystemNumURLs, predicate ) )
elif predicate_type == ClientSearch.PREDICATE_TYPE_SYSTEM_FILE_PROPERTIES:
recent_predicate_types = []
@ -742,10 +776,19 @@ class FleshOutPredicatePanel( ClientGUIScrolledPanels.EditPanel ):
recent_predicates = []
all_static_preds = set()
for pred_button in static_pred_buttons:
all_static_preds.update( pred_button.GetPredicates() )
if len( recent_predicate_types ) > 0:
recent_predicates = HG.client_controller.new_options.GetRecentPredicates( recent_predicate_types )
recent_predicates = [ pred for pred in recent_predicates if pred not in all_static_preds ]
if len( recent_predicates ) > 0:
recent_predicates_box = ClientGUICommon.StaticBox( page_panel, 'recent' )
@ -768,18 +811,6 @@ class FleshOutPredicatePanel( ClientGUIScrolledPanels.EditPanel ):
preds = button.GetPredicates()
if len( preds ) == 1:
pred = list( preds )[0]
if pred in recent_predicates:
button.setVisible( False )
continue
QP.AddToLayout( page_vbox, button, CC.FLAGS_EXPAND_PERPENDICULAR )
button.predicatesChosen.connect( self.StaticButtonClicked )
@ -791,6 +822,8 @@ class FleshOutPredicatePanel( ClientGUIScrolledPanels.EditPanel ):
QP.AddToLayout( page_vbox, panel, CC.FLAGS_EXPAND_PERPENDICULAR )
page_vbox.addStretch( 1 )
page_panel.setLayout( page_vbox )
if i == 0 and len( static_pred_buttons ) > 0 and len( editable_pred_panels ) == 0:

4
hydrus/client/importing/ClientImportFiles.py
View File

@ -356,7 +356,9 @@ class FileImportJob( object ):
percentage_in = HG.client_controller.new_options.GetInteger( 'video_thumbnail_percentage_in' )
thumbnail_numpy = HydrusFileHandling.GenerateThumbnailNumPy(self._temp_path, target_resolution, mime, duration, num_frames, percentage_in = percentage_in)
extra_description = f'File with hash "{self.GetHash().hex()}".'
thumbnail_numpy = HydrusFileHandling.GenerateThumbnailNumPy( self._temp_path, target_resolution, mime, duration, num_frames, percentage_in = percentage_in, extra_description = extra_description )
# this guy handles almost all his own exceptions now, so no need for clever catching. if it fails, we are prob talking an I/O failure, which is not a 'thumbnail failed' error
self._thumbnail_bytes = HydrusImageHandling.GenerateThumbnailBytesFromNumPy( thumbnail_numpy )

19
hydrus/client/media/ClientMedia.py
View File

@ -1150,7 +1150,7 @@ class MediaList( object ):
#
physically_deleted = service_key == CC.COMBINED_LOCAL_FILE_SERVICE_KEY
trashed = service_key in local_file_domains
possibly_trashed = service_key in local_file_domains and action == HC.CONTENT_UPDATE_DELETE
deleted_from_our_domain = self._location_context.IsOneDomain() and service_key in self._location_context.current_service_keys
we_are_looking_at_trash = self._location_context.IsOneDomain() and CC.TRASH_SERVICE_KEY in self._location_context.current_service_keys
@ -1162,12 +1162,21 @@ class MediaList( object ):
# case two, disappeared from repo hard drive while we are looking at it
deleted_from_repo_and_repo_view = service_key not in all_local_file_services and deleted_from_our_domain
# case three, user asked for this to happen
user_says_remove_and_trashed_from_non_trash_local_view = HC.options[ 'remove_trashed_files' ] and trashed and not we_are_looking_at_trash
user_says_remove_and_possibly_trashed_from_non_trash_local_view = HC.options[ 'remove_trashed_files' ] and possibly_trashed and not we_are_looking_at_trash
if physically_deleted_and_local_view or user_says_remove_and_trashed_from_non_trash_local_view or deleted_from_repo_and_repo_view:
if physically_deleted_and_local_view or user_says_remove_and_possibly_trashed_from_non_trash_local_view or deleted_from_repo_and_repo_view:
self._RemoveMediaByHashes( hashes )
if user_says_remove_and_possibly_trashed_from_non_trash_local_view:
actual_trash_hashes = self.GetHashes( is_in_file_service_key = CC.TRASH_SERVICE_KEY )
hashes = set( hashes ).intersection( actual_trash_hashes )
if len( hashes ) > 0:
self._RemoveMediaByHashes( hashes )

7
hydrus/client/metadata/ClientTagsHandling.py
View File

@ -870,11 +870,16 @@ class TagDisplayManager( HydrusSerialisable.SerialisableBase ):
def HideTag( self, tag_display_type, service_key, tag ):
self.HideTags( tag_display_type, service_key, ( tag, ) )
def HideTags( self, tag_display_type, service_key, tags ):
with self._lock:
tag_filter = self._tag_display_types_to_service_keys_to_tag_filters[ tag_display_type ][ service_key ]
tag_filter.SetRule( tag, HC.FILTER_BLACKLIST )
tag_filter.SetRules( tags, HC.FILTER_BLACKLIST )
self._dirty = True

52
hydrus/client/search/ClientSearch.py
View File

@ -71,6 +71,8 @@ PREDICATE_TYPE_SYSTEM_SIMILAR_TO_DATA = 48
PREDICATE_TYPE_SYSTEM_SIMILAR_TO = 49
PREDICATE_TYPE_SYSTEM_HAS_TRANSPARENCY = 50
PREDICATE_TYPE_SYSTEM_HAS_FORCED_FILETYPE = 51
PREDICATE_TYPE_SYSTEM_NUM_URLS = 52
PREDICATE_TYPE_SYSTEM_URLS = 53
SYSTEM_PREDICATE_TYPES = {
PREDICATE_TYPE_SYSTEM_EVERYTHING,
@ -116,6 +118,8 @@ SYSTEM_PREDICATE_TYPES = {
PREDICATE_TYPE_SYSTEM_FILE_RELATIONSHIPS_COUNT,
PREDICATE_TYPE_SYSTEM_FILE_RELATIONSHIPS_KING,
PREDICATE_TYPE_SYSTEM_KNOWN_URLS,
PREDICATE_TYPE_SYSTEM_NUM_URLS,
PREDICATE_TYPE_SYSTEM_URLS,
PREDICATE_TYPE_SYSTEM_FILE_VIEWING_STATS,
PREDICATE_TYPE_SYSTEM_TIME,
PREDICATE_TYPE_SYSTEM_HAS_FORCED_FILETYPE
@ -362,11 +366,15 @@ class NumberTest( HydrusSerialisable.SerialisableBase ):
return lambda x: lower < x < upper
elif self.operator == NUMBER_TEST_OPERATOR_NOT_EQUAL:
return lambda x: x != self.value
def IsAnythingButZero( self ):
return self.operator == NUMBER_TEST_OPERATOR_GREATER_THAN and self.value == 0
return self.operator in ( NUMBER_TEST_OPERATOR_NOT_EQUAL, NUMBER_TEST_OPERATOR_GREATER_THAN ) and self.value == 0
def IsZero( self ):
@ -390,6 +398,14 @@ class NumberTest( HydrusSerialisable.SerialisableBase ):
return NumberTest( operator, value )
@staticmethod
def STATICCreateMegaLambda( number_tests: typing.Collection[ "NumberTest" ] ):
lambdas = [ number_test.GetLambda() for number_test in number_tests ]
return lambda x: False not in ( lamb( x ) for lamb in lambdas )
HydrusSerialisable.SERIALISABLE_TYPES_TO_OBJECT_TYPES[ HydrusSerialisable.SERIALISABLE_TYPE_NUMBER_TEST ] = NumberTest
class FileSystemPredicates( object ):
@ -416,6 +432,7 @@ class FileSystemPredicates( object ):
self._ratings_predicates = []
self._num_tags_predicates = []
self._num_urls_predicates = []
self._duplicate_count_predicates = []
@ -697,6 +714,11 @@ class FileSystemPredicates( object ):
self._num_tags_predicates.append( predicate.Duplicate() )
if predicate_type == PREDICATE_TYPE_SYSTEM_NUM_URLS:
self._num_urls_predicates.append( predicate.Duplicate() )
if predicate_type == PREDICATE_TYPE_SYSTEM_TAG_AS_NUMBER:
( namespace, operator, num ) = value
@ -930,6 +952,22 @@ class FileSystemPredicates( object ):
return namespaces_to_tests
def GetNumURLsNumberTests( self ) -> typing.List[ NumberTest ]:
tests = []
for predicate in self._num_urls_predicates:
( operator, value ) = predicate.GetValue()
test = NumberTest.STATICCreateFromCharacters( operator, value )
tests.append( test )
return tests
def GetRatingsPredicates( self ):
return self._ratings_predicates
@ -1630,6 +1668,7 @@ EDIT_PRED_TYPES = {
PREDICATE_TYPE_SYSTEM_NUM_FRAMES,
PREDICATE_TYPE_SYSTEM_FILE_SERVICE,
PREDICATE_TYPE_SYSTEM_KNOWN_URLS,
PREDICATE_TYPE_SYSTEM_NUM_URLS,
PREDICATE_TYPE_SYSTEM_HASH,
PREDICATE_TYPE_SYSTEM_LIMIT,
PREDICATE_TYPE_SYSTEM_MIME,
@ -2155,7 +2194,7 @@ class Predicate( HydrusSerialisable.SerialisableBase ):
return Predicate( self._predicate_type, not self._value )
elif self._predicate_type in ( PREDICATE_TYPE_SYSTEM_NUM_NOTES, PREDICATE_TYPE_SYSTEM_NUM_WORDS, PREDICATE_TYPE_SYSTEM_NUM_FRAMES, PREDICATE_TYPE_SYSTEM_DURATION ):
elif self._predicate_type in ( PREDICATE_TYPE_SYSTEM_NUM_NOTES, PREDICATE_TYPE_SYSTEM_NUM_WORDS, PREDICATE_TYPE_SYSTEM_NUM_URLS, PREDICATE_TYPE_SYSTEM_NUM_FRAMES, PREDICATE_TYPE_SYSTEM_DURATION ):
( operator, value ) = self._value
@ -2413,9 +2452,10 @@ class Predicate( HydrusSerialisable.SerialisableBase ):
elif self._predicate_type == PREDICATE_TYPE_SYSTEM_DIMENSIONS: base = 'dimensions'
elif self._predicate_type == PREDICATE_TYPE_SYSTEM_SIMILAR_TO: base = 'similar files'
elif self._predicate_type == PREDICATE_TYPE_SYSTEM_TIME: base = 'time'
elif self._predicate_type == PREDICATE_TYPE_SYSTEM_URLS: base = 'urls'
elif self._predicate_type == PREDICATE_TYPE_SYSTEM_NOTES: base = 'notes'
elif self._predicate_type == PREDICATE_TYPE_SYSTEM_FILE_RELATIONSHIPS: base = 'file relationships'
elif self._predicate_type in ( PREDICATE_TYPE_SYSTEM_WIDTH, PREDICATE_TYPE_SYSTEM_HEIGHT, PREDICATE_TYPE_SYSTEM_NUM_NOTES, PREDICATE_TYPE_SYSTEM_NUM_WORDS, PREDICATE_TYPE_SYSTEM_NUM_FRAMES ):
elif self._predicate_type in ( PREDICATE_TYPE_SYSTEM_WIDTH, PREDICATE_TYPE_SYSTEM_HEIGHT, PREDICATE_TYPE_SYSTEM_NUM_NOTES, PREDICATE_TYPE_SYSTEM_NUM_URLS, PREDICATE_TYPE_SYSTEM_NUM_WORDS, PREDICATE_TYPE_SYSTEM_NUM_FRAMES ):
has_phrase = None
not_has_phrase = None
@ -2434,6 +2474,12 @@ class Predicate( HydrusSerialisable.SerialisableBase ):
has_phrase = ': has notes'
not_has_phrase = ': no notes'
elif self._predicate_type == PREDICATE_TYPE_SYSTEM_NUM_URLS:
base = 'number of urls'
has_phrase = ': has urls'
not_has_phrase = ': no urls'
elif self._predicate_type == PREDICATE_TYPE_SYSTEM_NUM_WORDS:
base = 'number of words'

1
hydrus/client/search/ClientSearchParseSystemPredicates.py
View File

@ -229,6 +229,7 @@ pred_generators = {
SystemPredicateParser.Predicate.UNTAGGED : lambda o, v, u: ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_TAGS, ( '*', '=', 0 ) ),
SystemPredicateParser.Predicate.NUM_OF_TAGS : lambda o, v, u: ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_TAGS, ( '*', o, v ) ),
SystemPredicateParser.Predicate.NUM_OF_TAGS_WITH_NAMESPACE : lambda o, v, u: ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_TAGS, v ),
SystemPredicateParser.Predicate.NUM_OF_URLS : lambda o, v, u: ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_URLS, ( o, v ) ),
SystemPredicateParser.Predicate.NUM_OF_WORDS : lambda o, v, u: ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_WORDS, ( o, v ) ),
SystemPredicateParser.Predicate.HEIGHT : lambda o, v, u: ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_HEIGHT, ( o, v ) ),
SystemPredicateParser.Predicate.WIDTH : lambda o, v, u: ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_WIDTH, ( o, v ) ),

2
hydrus/core/HydrusConstants.py
View File

@ -105,7 +105,7 @@ options = {}
# Misc
NETWORK_VERSION = 20
SOFTWARE_VERSION = 557
SOFTWARE_VERSION = 558
CLIENT_API_VERSION = 58
SERVER_THUMBNAIL_DIMENSIONS = ( 200, 200 )

32
hydrus/core/HydrusData.py
View File

@ -164,6 +164,37 @@ def ConvertIntToPrettyOrdinalString( num: int ):
return s
def ConvertManyStringsToNiceInsertableHumanSummary( texts: typing.Collection[ str ], do_sort: bool = True ) -> str:
"""
The purpose of this guy is to convert your list of 20 subscription names or whatever to something you can present to the user without making a giganto tall dialog.
"""
texts = list( texts )
if do_sort:
HydrusText.SortStringsIgnoringCase( texts )
if len( texts ) == 1:
return f' "{texts[0]}" '
else:
if len( texts ) <= 4:
t = '\n'.join( texts )
else:
t = ', '.join( texts )
return f'\n\n{t}\n\n'
def ConvertIntToUnit( unit ):
if unit == 1: return 'B'
@ -311,6 +342,7 @@ def GenerateKey():
return os.urandom( HC.HYDRUS_KEY_LENGTH )
def Get64BitHammingDistance( perceptual_hash1, perceptual_hash2 ):
# old slow strategy:

1
hydrus/core/HydrusSerialisable.py
View File

@ -9,7 +9,6 @@ from hydrus.core import HydrusCompression
from hydrus.core import HydrusConstants as HC
from hydrus.core import HydrusData
from hydrus.core import HydrusExceptions
from hydrus.core import HydrusTime
META_SERIALISABLE_TYPE_JSON_OK = 0
META_SERIALISABLE_TYPE_JSON_BYTES = 1

7
hydrus/core/HydrusTags.py
View File

@ -731,12 +731,7 @@ class TagFilter( HydrusSerialisable.SerialisableBase ):
def SetRule( self, tag_slice, rule ):
with self._lock:
self._tag_slices_to_rules[ tag_slice ] = rule
self._UpdateRuleCache()
self.SetRules( ( tag_slice, ), rule )
def SetRules( self, tag_slices, rule ):

56
hydrus/core/files/HydrusArchiveHandling.py
View File

@ -135,17 +135,20 @@ def ZipLooksLikeCBZ( path_to_zip ):
# what does a Comic Book Archive look like? it is ad-hoc, not rigorous, so be forgiving
# it is a list of images
# they may be flat in the base, or they may be in one or more subfolders
# they may be accomanied by extra metadata files like: md5sum, .sfv/.SFV, .nfo, comicbook.xml, metadata.txt
# they may be accompanied by extra metadata files like: md5sum, .sfv/.SFV, .nfo, comicbook.xml, metadata.txt
# they _cannot_ be accompanied by .exe, .zip, .unitypackage, .psd, or other gubbins
# nothing else
directories_to_image_filenames = collections.defaultdict( set )
num_directories = 1
directories_with_stuff_in = set()
num_weird_files = 0
num_images = 0
num_images_with_numbers = 0
num_weird_files_allowed_per_directory = 5
num_images_needed_per_directory = 1
ok_weird_filenames = { 'md5sum', 'comicbook.xml', 'metadata.txt', 'info.txt' }
totally_ok_weird_filenames = { 'md5sum', 'comicbook.xml', 'metadata.txt', 'info.txt' }
weird_filename_extension_whitelist = { '.sfv', '.nfo', '.txt', '.xml', '.json' }
with zipfile.ZipFile( path_to_zip ) as zip_handle:
@ -153,8 +156,6 @@ def ZipLooksLikeCBZ( path_to_zip ):
if zip_info.is_dir():
num_directories += 1
continue
@ -171,9 +172,11 @@ def ZipLooksLikeCBZ( path_to_zip ):
directory_path = ''
directories_with_stuff_in.add( directory_path )
filename = filename.lower()
if filename in ok_weird_filenames:
if filename in totally_ok_weird_filenames:
continue
@ -182,21 +185,46 @@ def ZipLooksLikeCBZ( path_to_zip ):
num_images += 1
if re.search( r'\d', filename ) is not None:
num_images_with_numbers += 1
directories_to_image_filenames[ directory_path ].add( filename )
continue
elif filename_has_video_ext( filename ):
# this catches some zips nicely
return False
else:
num_weird_files += 1
if '.' in filename:
ext_with_dot = '.' + filename.split( '.' )[-1]
if ext_with_dot in weird_filename_extension_whitelist:
num_weird_files += 1
else:
# we got ourselves a .mp4 or .unitypackage or whatever. not a cbz!
return False
else:
# we out here with a 'gonk' file. not a cbz!
return False
# although we want to broadly check there are files with numbers, we don't want tempt ourselves into searching for 0 or 1. some 'chapters' start at page 55
# a two-paged comic is the minimum permissible
if num_images_with_numbers <= 1:
return False
try:
path = GetCoverPagePath( zip_handle )
@ -241,12 +269,12 @@ def ZipLooksLikeCBZ( path_to_zip ):
if num_weird_files * num_directories > num_weird_files_allowed_per_directory:
if num_weird_files * len( directories_with_stuff_in ) > num_weird_files_allowed_per_directory:
return False
if num_images * num_directories < num_images_needed_per_directory:
if num_images * len( directories_with_stuff_in ) < num_images_needed_per_directory:
return False

73
hydrus/core/files/HydrusFileHandling.py
View File

@ -1,5 +1,6 @@
import hashlib
import os
import typing
from hydrus.core import HydrusConstants as HC
from hydrus.core import HydrusData
@ -50,7 +51,22 @@ def GenerateThumbnailBytes( path, target_resolution, mime, duration, num_frames,
return HydrusImageHandling.GenerateThumbnailBytesFromNumPy( thumbnail_numpy )
def GenerateThumbnailNumPy( path, target_resolution, mime, duration, num_frames, percentage_in = 35 ):
def PrintMoreThumbErrorInfo( e: Exception, message, extra_description: typing.Optional[ str ] = None ):
if not isinstance( e, HydrusExceptions.NoThumbnailFileException ):
HydrusData.Print( message )
if extra_description is not None:
HydrusData.Print( f'Extra info: {extra_description}' )
HydrusData.PrintException( e )
def GenerateThumbnailNumPy( path, target_resolution, mime, duration, num_frames, percentage_in = 35, extra_description = None ):
if mime == HC.APPLICATION_CBZ:
@ -64,7 +80,9 @@ def GenerateThumbnailNumPy( path, target_resolution, mime, duration, num_frames,
thumbnail_numpy = HydrusImageHandling.GenerateThumbnailNumPyFromStaticImagePath( temp_path, target_resolution, cover_mime )
except:
except Exception as e:
PrintMoreThumbErrorInfo( e, f'Problem generating thumbnail for "{path}".', extra_description = extra_description )
thumb_path = os.path.join( HC.STATIC_DIR, 'zip.png' )
@ -85,7 +103,9 @@ def GenerateThumbnailNumPy( path, target_resolution, mime, duration, num_frames,
thumbnail_numpy = HydrusImageHandling.GenerateThumbnailNumPyFromStaticImagePath( temp_path, target_resolution, HC.IMAGE_PNG )
except:
except Exception as e:
PrintMoreThumbErrorInfo( e, f'Problem generating thumbnail for "{path}".', extra_description = extra_description )
thumb_path = os.path.join( HC.STATIC_DIR, 'clip.png' )
@ -104,11 +124,7 @@ def GenerateThumbnailNumPy( path, target_resolution, mime, duration, num_frames,
except Exception as e:
if not isinstance( e, HydrusExceptions.NoThumbnailFileException ):
HydrusData.Print( 'Problem generating thumbnail for "{}":'.format( path ) )
HydrusData.PrintException( e )
PrintMoreThumbErrorInfo( e, f'Problem generating thumbnail for "{path}".', extra_description = extra_description )
thumb_path = os.path.join( HC.STATIC_DIR, 'krita.png' )
@ -127,6 +143,8 @@ def GenerateThumbnailNumPy( path, target_resolution, mime, duration, num_frames,
except Exception as e:
PrintMoreThumbErrorInfo( e, f'Problem generating thumbnail for "{path}".', extra_description = extra_description )
thumb_path = os.path.join( HC.STATIC_DIR, 'procreate.png' )
thumbnail_numpy = HydrusImageHandling.GenerateThumbnailNumPyFromStaticImagePath( thumb_path, target_resolution, HC.IMAGE_PNG )
@ -144,8 +162,8 @@ def GenerateThumbnailNumPy( path, target_resolution, mime, duration, num_frames,
except Exception as e:
HydrusData.Print( 'Problem generating thumbnail for "{}":'.format( path ) )
HydrusData.PrintException( e )
PrintMoreThumbErrorInfo( e, f'Problem generating thumbnail for "{path}".', extra_description = extra_description )
HydrusData.Print( 'Attempting ffmpeg PSD thumbnail fallback' )
( os_file_handle, temp_path ) = HydrusTemp.GetTempPath( suffix = '.png' )
@ -158,6 +176,8 @@ def GenerateThumbnailNumPy( path, target_resolution, mime, duration, num_frames,
except Exception as e:
PrintMoreThumbErrorInfo( e, f'Secondary problem generating thumbnail for "{path}".', extra_description = extra_description )
thumb_path = os.path.join( HC.STATIC_DIR, 'psd.png' )
thumbnail_numpy = HydrusImageHandling.GenerateThumbnailNumPyFromStaticImagePath( thumb_path, target_resolution, HC.IMAGE_PNG )
@ -176,11 +196,7 @@ def GenerateThumbnailNumPy( path, target_resolution, mime, duration, num_frames,
except Exception as e:
if not isinstance( e, HydrusExceptions.NoThumbnailFileException ):
HydrusData.Print( 'Problem generating thumbnail for "{}":'.format( path ) )
HydrusData.PrintException( e )
PrintMoreThumbErrorInfo( e, f'Problem generating thumbnail for "{path}".', extra_description = extra_description )
thumb_path = os.path.join( HC.STATIC_DIR, 'svg.png' )
@ -195,11 +211,7 @@ def GenerateThumbnailNumPy( path, target_resolution, mime, duration, num_frames,
except Exception as e:
if not isinstance( e, HydrusExceptions.NoThumbnailFileException ):
HydrusData.Print( 'Problem generating thumbnail for "{}":'.format( path ) )
HydrusData.PrintException( e )
PrintMoreThumbErrorInfo( e, f'Problem generating thumbnail for "{path}".', extra_description = extra_description )
thumb_path = os.path.join( HC.STATIC_DIR, 'pdf.png' )
@ -216,7 +228,9 @@ def GenerateThumbnailNumPy( path, target_resolution, mime, duration, num_frames,
thumbnail_numpy = HydrusImageHandling.GenerateThumbnailNumPyFromStaticImagePath( temp_path, target_resolution, HC.IMAGE_PNG )
except:
except Exception as e:
PrintMoreThumbErrorInfo( e, f'Problem generating thumbnail for "{path}".', extra_description = extra_description )
thumb_path = os.path.join( HC.STATIC_DIR, 'flash.png' )
@ -237,8 +251,7 @@ def GenerateThumbnailNumPy( path, target_resolution, mime, duration, num_frames,
except Exception as e:
HydrusData.Print( 'Problem generating thumbnail for "{}":'.format( path ) )
HydrusData.PrintException( e )
PrintMoreThumbErrorInfo( e, f'Problem generating thumbnail for "{path}".', extra_description = extra_description )
thumb_path = os.path.join( HC.STATIC_DIR, 'hydrus.png' )
@ -259,7 +272,9 @@ def GenerateThumbnailNumPy( path, target_resolution, mime, duration, num_frames,
thumbnail_numpy = HydrusImageHandling.GenerateThumbnailNumPyFromStaticImagePath( temp_path, target_resolution, cover_mime )
except:
except Exception as e:
PrintMoreThumbErrorInfo( e, f'Problem generating thumbnail for "{path}".', extra_description = extra_description )
thumb_path = os.path.join( HC.STATIC_DIR, 'zip.png' )
@ -284,8 +299,9 @@ def GenerateThumbnailNumPy( path, target_resolution, mime, duration, num_frames,
except Exception as e:
HydrusData.Print( 'Problem generating thumbnail for "{}" at frame {} ({})--FFMPEG could not render it.'.format( path, desired_thumb_frame_index, HydrusData.ConvertFloatToPercentage( percentage_in / 100.0 ) ) )
HydrusData.PrintException( e )
message = 'Problem generating thumbnail for "{}" at frame {} ({})--FFMPEG could not render it.'.format( path, desired_thumb_frame_index, HydrusData.ConvertFloatToPercentage( percentage_in / 100.0 ) )
PrintMoreThumbErrorInfo( e, message, extra_description = extra_description )
numpy_image = None
@ -307,8 +323,9 @@ def GenerateThumbnailNumPy( path, target_resolution, mime, duration, num_frames,
except Exception as e:
HydrusData.Print( 'Problem generating thumbnail for "{}" at first frame--FFMPEG could not render it.'.format( path ) )
HydrusData.PrintException( e )
message = 'Problem generating thumbnail for "{}" at first frame--FFMPEG could not render it.'.format( path )
PrintMoreThumbErrorInfo( e, message, extra_description = extra_description )
numpy_image = None

25
hydrus/external/SystemPredicateParser.py vendored
View File

@ -108,6 +108,7 @@ class Predicate( Enum ):
UNTAGGED = auto()
NUM_OF_TAGS = auto()
NUM_OF_TAGS_WITH_NAMESPACE = auto()
NUM_OF_URLS = auto()
NUM_OF_WORDS = auto()
HEIGHT = auto()
WIDTH = auto()
@ -234,7 +235,8 @@ SYSTEM_PREDICATES = {
'untagged|no tags': (Predicate.UNTAGGED, None, None, None),
'num(ber)?( of)? tags': (Predicate.NUM_OF_TAGS, Operators.RELATIONAL, Value.NATURAL, None),
'num(ber)?( of)? (?=[^\\s].* tags)': (Predicate.NUM_OF_TAGS_WITH_NAMESPACE, None, Value.NAMESPACE_AND_NUM_TAGS, None),
'num(ber)?( of)? words': (Predicate.NUM_OF_WORDS, Operators.RELATIONAL, Value.NATURAL, None),
'num(ber)?( of)? urls': (Predicate.NUM_OF_URLS, Operators.RELATIONAL, Value.NATURAL, None),
'num(ber)?( of)? words': (Predicate.NUM_OF_WORDS, Operators.RELATIONAL_EXACT, Value.NATURAL, None),
'height': (Predicate.HEIGHT, Operators.RELATIONAL, Value.NATURAL, Units.PIXELS_OR_NONE),
'width': (Predicate.WIDTH, Operators.RELATIONAL, Value.NATURAL, Units.PIXELS_OR_NONE),
'file ?size': (Predicate.FILESIZE, Operators.RELATIONAL, Value.NATURAL, Units.FILESIZE),
@ -295,6 +297,9 @@ def string_looks_like_date( string ):
# then trying to parse it by consuming the input string.
# The parse_* functions consume some of the string and return a (remaining part of the string, parsed value) tuple.
def parse_system_predicate( string: str ):
# TODO: (hydev): rework this thing into passing around a 'parse result object' that the operator parser can set a value for and say 'yeah value is sorted' for things like 'has words' = '> 0' in one swoop
string = string.lower().strip()
string = string.replace( '_', ' ' )
if string.startswith( "-" ):
@ -370,13 +375,21 @@ def parse_unit( string: str, spec ):
def parse_value( string: str, spec ):
string = string.strip()
if spec is None:
return string, None
elif spec in ( Value.NATURAL, Value.INTEGER ):
# 'has urls', 'has words'
if string.startswith( 'has' ) or string.startswith( 'no' ):
return '', 0
match = re.match( '-?[0-9,]+', string )
if match:
@ -472,10 +485,13 @@ def parse_value( string: str, spec ):
elif spec == Value.FILETYPE_LIST:
valid_values = sorted( FILETYPES.keys(), key = lambda k: len( k ), reverse = True )
ftype_regex = '(' + '|'.join( [ '(' + val + ')' for val in valid_values ] ) + ')'
match = re.match( '(' + ftype_regex + '(\s|,)+)*' + ftype_regex, string )
if match:
found_ftypes_all = re.sub( '\s', ' ', match[ 0 ].replace( ',', '|' ) ).split( '|' )
found_ftypes_good = [ ]
for ftype in found_ftypes_all:
@ -483,7 +499,10 @@ def parse_value( string: str, spec ):
if len( ftype ) > 0 and ftype in FILETYPES:
found_ftypes_good.extend( FILETYPES[ ftype ] )
return string[ len( match[ 0 ] ): ], set( found_ftypes_good )
raise ValueError( "Invalid value, expected a list of file types" )
elif spec == Value.DATE_OR_TIME_INTERVAL:
if DATEPARSER_OK:
@ -535,6 +554,8 @@ def parse_value( string: str, spec ):
return string_result, (years, months, days, hours)
match = re.match( '(?P<year>[0-9][0-9][0-9][0-9])-(?P<month>[0-9][0-9]?)-(?P<day>[0-9][0-9]?)', string )
if match:
# good expansion here would be to parse a full date with 08:20am kind of thing, but we'll wait for better datetime parsing library for that I think!
@ -748,6 +769,8 @@ def parse_operator( string: str, spec ):
for op in ops:
if string.startswith( op ): return string[ len( op ): ], op
if string.startswith( 'is' ): return string[ 2: ], '='
if string.startswith( 'has' ): return string, '>'
if string.startswith( 'no' ): return string, '='
raise ValueError( "Invalid relational operator" )
elif spec == Operators.RELATIONAL_FOR_RATING_SERVICE:

9
hydrus/test/TestClientDB.py
View File

@ -480,6 +480,13 @@ class TestClientDB( unittest.TestCase ):
tests.append( ( ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_WORDS, ( '>', 0 ), 0 ) )
tests.append( ( ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_WORDS, ( '>', 1 ), 0 ) )
tests.append( ( ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_URLS, ( '<', 1 ), 1 ) )
tests.append( ( ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_URLS, ( '<', 0 ), 0 ) )
tests.append( ( ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_URLS, ( '=', 0 ), 1 ) )
tests.append( ( ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_URLS, ( '=', 1 ), 0 ) )
tests.append( ( ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_URLS, ( '>', 0 ), 0 ) )
tests.append( ( ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_URLS, ( '>', 1 ), 0 ) )
tests.append( ( ClientSearch.PREDICATE_TYPE_SYSTEM_RATIO, ( '=', 1, 1 ), 1 ) )
tests.append( ( ClientSearch.PREDICATE_TYPE_SYSTEM_RATIO, ( '=', 4, 3 ), 0 ) )
tests.append( ( ClientSearch.PREDICATE_TYPE_SYSTEM_RATIO, ( HC.UNICODE_APPROX_EQUAL, 1, 1 ), 1 ) )
@ -808,7 +815,7 @@ class TestClientDB( unittest.TestCase ):
predicates.append( ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_EVERYTHING, count = ClientSearch.PredicateCount.STATICCreateCurrentCount( 1 ) ) )
predicates.append( ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_INBOX, count = ClientSearch.PredicateCount.STATICCreateCurrentCount( 1 ) ) )
predicates.append( ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_ARCHIVE, count = ClientSearch.PredicateCount.STATICCreateCurrentCount( 0 ) ) )
predicates.extend( [ ClientSearch.Predicate( predicate_type ) for predicate_type in [ ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_TAGS, ClientSearch.PREDICATE_TYPE_SYSTEM_LIMIT, ClientSearch.PREDICATE_TYPE_SYSTEM_SIZE, ClientSearch.PREDICATE_TYPE_SYSTEM_TIME, ClientSearch.PREDICATE_TYPE_SYSTEM_KNOWN_URLS, ClientSearch.PREDICATE_TYPE_SYSTEM_FILE_PROPERTIES, ClientSearch.PREDICATE_TYPE_SYSTEM_HASH, ClientSearch.PREDICATE_TYPE_SYSTEM_DIMENSIONS, ClientSearch.PREDICATE_TYPE_SYSTEM_DURATION, ClientSearch.PREDICATE_TYPE_SYSTEM_NOTES, ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_WORDS, ClientSearch.PREDICATE_TYPE_SYSTEM_MIME, ClientSearch.PREDICATE_TYPE_SYSTEM_RATING, ClientSearch.PREDICATE_TYPE_SYSTEM_SIMILAR_TO, ClientSearch.PREDICATE_TYPE_SYSTEM_FILE_SERVICE, ClientSearch.PREDICATE_TYPE_SYSTEM_TAG_AS_NUMBER, ClientSearch.PREDICATE_TYPE_SYSTEM_FILE_RELATIONSHIPS, ClientSearch.PREDICATE_TYPE_SYSTEM_FILE_VIEWING_STATS ] ] )
predicates.extend( [ ClientSearch.Predicate( predicate_type ) for predicate_type in [ ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_TAGS, ClientSearch.PREDICATE_TYPE_SYSTEM_LIMIT, ClientSearch.PREDICATE_TYPE_SYSTEM_SIZE, ClientSearch.PREDICATE_TYPE_SYSTEM_TIME, ClientSearch.PREDICATE_TYPE_SYSTEM_URLS, ClientSearch.PREDICATE_TYPE_SYSTEM_FILE_PROPERTIES, ClientSearch.PREDICATE_TYPE_SYSTEM_HASH, ClientSearch.PREDICATE_TYPE_SYSTEM_DIMENSIONS, ClientSearch.PREDICATE_TYPE_SYSTEM_DURATION, ClientSearch.PREDICATE_TYPE_SYSTEM_NOTES, ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_WORDS, ClientSearch.PREDICATE_TYPE_SYSTEM_MIME, ClientSearch.PREDICATE_TYPE_SYSTEM_RATING, ClientSearch.PREDICATE_TYPE_SYSTEM_SIMILAR_TO, ClientSearch.PREDICATE_TYPE_SYSTEM_FILE_SERVICE, ClientSearch.PREDICATE_TYPE_SYSTEM_TAG_AS_NUMBER, ClientSearch.PREDICATE_TYPE_SYSTEM_FILE_RELATIONSHIPS, ClientSearch.PREDICATE_TYPE_SYSTEM_FILE_VIEWING_STATS ] ] )
self.assertEqual( set( result ), set( predicates ) )

19
hydrus/test/TestClientTags.py
View File

@ -1913,6 +1913,12 @@ class TestTagObjects( unittest.TestCase ):
self.assertEqual( p.GetNamespace(), 'system' )
self.assertEqual( p.GetTextsAndNamespaces( render_for_user ), [ ( p.ToString(), 'namespace', p.GetNamespace() ) ] )
p = ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_URLS, ( '<', 5 ) )
self.assertEqual( p.ToString(), 'system:number of urls < 5' )
self.assertEqual( p.GetNamespace(), 'system' )
self.assertEqual( p.GetTextsAndNamespaces( render_for_user ), [ ( p.ToString(), 'namespace', p.GetNamespace() ) ] )
p = ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_NUM_WORDS, ( '<', 5000 ) )
self.assertEqual( p.ToString(), 'system:number of words < 5,000' )
@ -2059,7 +2065,18 @@ class TestTagObjects( unittest.TestCase ):
( 'system:number of character tags > 5', "system:number of character tags > 5" ),
( f'system:number of tags {HC.UNICODE_APPROX_EQUAL} 10', "system:number of tags ~= 10" ),
( 'system:has tags', "system:number of tags > 0 " ),
( 'system:number of words < 2', "system:number of words < 2" ),
( 'system:number of urls < 2', 'system:number of urls < 2' ),
( 'system:number of urls < 2', 'system:num urls < 2' ),
( 'system:number of urls: has urls', 'system:num urls > 0' ),
( 'system:number of urls: has urls', 'system:num urls: has urls' ),
( 'system:number of urls: no urls', 'system:num urls = 0' ),
( 'system:number of urls: no urls', 'system:number of urls: no urls' ),
( 'system:number of urls < 2', 'system:number of urls < 2' ),
( 'system:number of urls < 2', 'system:num urls < 2' ),
( 'system:number of words: has words', 'system:num words > 0' ),
( 'system:number of words: has words', 'system:num words: has words' ),
( 'system:number of words: no words', 'system:num words = 0' ),
( 'system:number of words: no words', 'system:number of words: no words' ),
( 'system:height = 600', "system:height = 600px" ),
( 'system:height = 800', "system:height is 800" ),
( 'system:height > 900', "system:height > 900" ),