RepoMirrors
/
hydrus
mirror of https://github.com/hydrusnetwork/hydrus.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Version 242

This commit is contained in:
Hydrus Network Developer 2017-01-25 16:56:55 -06:00
parent e24ade0548
commit 595f64a7f1
36 changed files with 1219 additions and 653 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

35
help/changelog.html
View File

@ -8,6 +8,41 @@
<div class="content">
<h3>changelog</h3>
<ul>
<li><h3>version 242</h3></li>
<ul>
<li>optimised 'exact match' similar file queries to run a lot faster</li>
<li>optimised similar file queries in general, particularly for larger cycle queries</li>
<li>optimised hamming distance calculation, decreasing time by roughly 45%!</li>
<li>the similar files tree maintenance idle job will not trigger while there are phashes still to regenerate (this was redundantly and annoyingly blatting the new dupes page as soon as phash regen was paused)</li>
<li>removed similar files tree maintenance entry from db->maintain menu, as it can be done better from the new dupes page</li>
<li>adjusted the duplicate search and file phash regen progress gauges to reflect total number of files in cache, not the current batch job</li>
<li>all maintenance jobs on the duplicates search page will now save their progress (and free up a hanging gui) every 30 seconds</li>
<li>the duplicates page's cog menu button now lets you put phash regen and tree rebalancing on the normal idle routine, defaulting both to off</li>
<li>the cog menu can also put duplicate searching on idle time!</li>
<li>added a very rough 'just show me some pairs!' button to the dupe page--it is pretty neat to finally see what is going on</li>
<li>I may have reduced the memory use explosion some users are getting during file phash regen maintenance</li>
<li>wrote an unclose_page action and added it to the shortcuts options panel--it undoes the last page close, if one exists. ctrl+u will be the default for new users, but existing users have to add it under options</li>
<li>added ascending/descending sort choices for width, height, ratio, and num_pixels</li>
<li>the client can no longer talk to old http hydrus network servers--everything is now https</li>
<li>in prep for a later network version update, the client now supports gzipped network strings (which compress json a lot better than the old lz4 compression)</li>
<li>fixed gif rendering in the Windows build--I forgot to update a build script dll patch for the new version of opencv</li>
<li>the export file dialog's neighbouring .txt taglist file stuff now allows you to select a specific combination of tag services</li>
<li>if an hdd import's original file is due to be deleted, any existing neighbouring taglist .txt file will now also be deleted</li>
<li>the inter-thread messaging system has a new simple way of reporting download progress on an url</li>
<li>the handful of things that create a downloading popup (like the youtube downloader) now use this new download reporting system</li>
<li>sankaku seems to be 503-broke due to cloudflare protection--I have paused all existing sankaku subscriptions and removed the sankaku entry for new users (pending a future fix on my or their end)</li>
<li>I've also removed danbooru for new users for now--someone can fix the long-running sample size file issue in the new downloader engine</li>
<li>removed unnamespaced tag support from the hentai-foundry parser--maybe someone can try to fix that mess in the new downloader engine</li>
<li>menubuttons can now handle boolean check menu items that are tied straight into hydrus's options</li>
<li>menus launched from the newer frame and dialog code will now correctly display their help text on the main gui frame's statusbar! (at least on Windows! Linux and OS X remain borked!)</li>
<li>fixed a unicode error parsing bug in the gallery downloader</li>
<li>the server stop (or restart) command now correctly uses https!</li>
<li>the server test code now works on https as appropriate</li>
<li>fixed some misc server test code</li>
<li>misc fixes</li>
<li>misc cleanup</li>
<li>misc layout cleanup</li>
</ul>
<li><h3>version 241</h3></li>
<ul>
<li>fixed the 'setnondupename' problem that was affecting 'add' actions on manage subscriptions, scripts, and import/export folders</li>

47
include/ClientConstants.py
View File

@ -249,6 +249,14 @@ SORT_BY_LEXICOGRAPHIC_NAMESPACE_ASC = 12
SORT_BY_LEXICOGRAPHIC_NAMESPACE_DESC = 13
SORT_BY_INCIDENCE_NAMESPACE_ASC = 14
SORT_BY_INCIDENCE_NAMESPACE_DESC = 15
SORT_BY_WIDTH_ASC = 16
SORT_BY_WIDTH_DESC = 17
SORT_BY_HEIGHT_ASC = 18
SORT_BY_HEIGHT_DESC = 19
SORT_BY_RATIO_ASC = 20
SORT_BY_RATIO_DESC = 21
SORT_BY_NUM_PIXELS_ASC = 22
SORT_BY_NUM_PIXELS_DESC = 23
SORT_CHOICES = []
@ -258,30 +266,35 @@ SORT_CHOICES.append( ( 'system', SORT_BY_SHORTEST ) )
SORT_CHOICES.append( ( 'system', SORT_BY_LONGEST ) )
SORT_CHOICES.append( ( 'system', SORT_BY_NEWEST ) )
SORT_CHOICES.append( ( 'system', SORT_BY_OLDEST ) )
SORT_CHOICES.append( ( 'system', SORT_BY_WIDTH_ASC ) )
SORT_CHOICES.append( ( 'system', SORT_BY_WIDTH_DESC ) )
SORT_CHOICES.append( ( 'system', SORT_BY_HEIGHT_ASC ) )
SORT_CHOICES.append( ( 'system', SORT_BY_HEIGHT_DESC ) )
SORT_CHOICES.append( ( 'system', SORT_BY_RATIO_DESC ) )
SORT_CHOICES.append( ( 'system', SORT_BY_RATIO_ASC ) )
SORT_CHOICES.append( ( 'system', SORT_BY_NUM_PIXELS_ASC ) )
SORT_CHOICES.append( ( 'system', SORT_BY_NUM_PIXELS_DESC ) )
SORT_CHOICES.append( ( 'system', SORT_BY_MIME ) )
SORT_CHOICES.append( ( 'system', SORT_BY_RANDOM ) )
sort_enum_lookup = {}
sort_enum_lookup[ 'smallest first' ] = SORT_BY_SMALLEST
sort_enum_lookup[ 'largest first' ] = SORT_BY_LARGEST
sort_enum_lookup[ 'shortest first' ] = SORT_BY_SHORTEST
sort_enum_lookup[ 'longest first' ] = SORT_BY_LONGEST
sort_enum_lookup[ 'newest first' ] = SORT_BY_NEWEST
sort_enum_lookup[ 'oldest first' ] = SORT_BY_OLDEST
sort_enum_lookup[ 'order by mime' ] = SORT_BY_MIME
sort_enum_lookup[ 'random order' ] = SORT_BY_RANDOM
sort_string_lookup = {}
sort_string_lookup[ SORT_BY_SMALLEST ] = 'smallest first'
sort_string_lookup[ SORT_BY_LARGEST ] = 'largest first'
sort_string_lookup[ SORT_BY_SHORTEST ] = 'shortest first'
sort_string_lookup[ SORT_BY_LONGEST ] = 'longest first'
sort_string_lookup[ SORT_BY_NEWEST ] = 'newest first'
sort_string_lookup[ SORT_BY_OLDEST ] = 'oldest first'
sort_string_lookup[ SORT_BY_SMALLEST ] = 'smallest filesize first'
sort_string_lookup[ SORT_BY_LARGEST ] = 'largest filesize first'
sort_string_lookup[ SORT_BY_SHORTEST ] = 'shortest duration first'
sort_string_lookup[ SORT_BY_LONGEST ] = 'longest duration first'
sort_string_lookup[ SORT_BY_NEWEST ] = 'most recently imported first'
sort_string_lookup[ SORT_BY_OLDEST ] = 'least recently imported first'
sort_string_lookup[ SORT_BY_MIME ] = 'mime'
sort_string_lookup[ SORT_BY_RANDOM ] = 'random order'
sort_string_lookup[ SORT_BY_WIDTH_ASC ] = 'least wide first'
sort_string_lookup[ SORT_BY_WIDTH_DESC ] = 'most wide first'
sort_string_lookup[ SORT_BY_HEIGHT_ASC ] = 'least tall first'
sort_string_lookup[ SORT_BY_HEIGHT_DESC ] = 'most tall first'
sort_string_lookup[ SORT_BY_RATIO_ASC ] = 'tallest ratio first'
sort_string_lookup[ SORT_BY_RATIO_DESC ] = 'widest ratio first'
sort_string_lookup[ SORT_BY_NUM_PIXELS_ASC ] = 'fewest pixels first'
sort_string_lookup[ SORT_BY_NUM_PIXELS_DESC ] = 'most pixels first'
STATUS_UNKNOWN = 0
STATUS_SUCCESSFUL = 1

47
include/ClientController.py
View File

@ -87,13 +87,13 @@ class Controller( HydrusController.HydrusController ):
def CallBlockingToWx( self, callable, *args, **kwargs ):
def CallBlockingToWx( self, func, *args, **kwargs ):
def wx_code( job_key ):
try:
result = callable( *args, **kwargs )
result = func( *args, **kwargs )
job_key.SetVariable( 'result', result )
@ -479,6 +479,11 @@ class Controller( HydrusController.HydrusController ):
self.pub( 'refresh_status' )
def GetApp( self ):
return self._app
def GetClientFilesManager( self ):
return self._client_files_manager
@ -692,7 +697,43 @@ class Controller( HydrusController.HydrusController ):
loaded_into_disk_cache = HydrusGlobals.client_controller.Read( 'load_into_disk_cache', stop_time = disk_cache_stop_time, caller_limit = disk_cache_maintenance_mb * 1024 * 1024 )
self.WriteInterruptable( 'maintain_similar_files_tree', stop_time = stop_time )
if self._new_options.GetBoolean( 'maintain_similar_files_phashes_during_idle' ):
phashes_stop_time = stop_time
if phashes_stop_time is None:
phashes_stop_time = HydrusData.GetNow() + 15
self.WriteInterruptable( 'maintain_similar_files_phashes', stop_time = phashes_stop_time )
if self._new_options.GetBoolean( 'maintain_similar_files_tree_during_idle' ):
tree_stop_time = stop_time
if tree_stop_time is None:
tree_stop_time = HydrusData.GetNow() + 30
self.WriteInterruptable( 'maintain_similar_files_tree', stop_time = tree_stop_time, abandon_if_other_work_to_do = True )
if self._new_options.GetBoolean( 'maintain_similar_files_duplicate_pairs_during_idle' ):
search_distance = self._new_options.GetInteger( 'similar_files_duplicate_pairs_search_distance' )
search_stop_time = stop_time
if search_stop_time is None:
search_stop_time = HydrusData.GetNow() + 60
self.WriteInterruptable( 'maintain_similar_files_duplicate_pairs', search_distance, stop_time = search_stop_time, abandon_if_other_work_to_do = True )
self.WriteInterruptable( 'vacuum', stop_time = stop_time )

278
include/ClientDB.py
View File

@ -6,6 +6,7 @@ import ClientMedia
import ClientRatings
import ClientThreading
import collections
import gc
import hashlib
import httplib
import itertools
@ -24,7 +25,6 @@ import HydrusTagArchive
import HydrusTags
import HydrusThreading
import ClientConstants as CC
import lz4
import numpy
import os
import psutil
@ -1538,7 +1538,7 @@ class DB( HydrusDB.HydrusDB ):
( ancestor_phash, ancestor_radius, ancestor_inner_id, ancestor_inner_population, ancestor_outer_id, ancestor_outer_population ) = self._c.execute( 'SELECT phash, radius, inner_id, inner_population, outer_id, outer_population FROM shape_perceptual_hashes, shape_vptree USING ( phash_id ) WHERE phash_id = ?;', ( ancestor_id, ) ).fetchone()
distance_to_ancestor = HydrusData.GetHammingDistance( phash, ancestor_phash )
distance_to_ancestor = HydrusData.Get64BitHammingDistance( phash, ancestor_phash )
if ancestor_radius is None or distance_to_ancestor <= ancestor_radius:
@ -1681,7 +1681,7 @@ class DB( HydrusDB.HydrusDB ):
else:
children = [ ( HydrusData.GetHammingDistance( phash, child_phash ), child_id, child_phash ) for ( child_id, child_phash ) in children ]
children = [ ( HydrusData.Get64BitHammingDistance( phash, child_phash ), child_id, child_phash ) for ( child_id, child_phash ) in children ]
children.sort()
@ -1772,7 +1772,48 @@ class DB( HydrusDB.HydrusDB ):
return ( searched_distances_to_count, duplicate_types_to_count, num_phashes_to_regen, num_branches_to_regen )
def _CacheSimilarFilesMaintainDuplicatePairs( self, search_distance, job_key = None, stop_time = None ):
def _CacheSimilarFilesGetSomeDupes( self ):
result = self._c.execute( 'SELECT smaller_hash_id, larger_hash_id FROM duplicate_pairs WHERE duplicate_type = ? ORDER BY RANDOM() LIMIT 1;', ( HC.DUPLICATE_UNKNOWN, ) ).fetchone()
if result is None:
return set()
( random_hash_id, ) = random.sample( result, 1 ) # either the smaller or larger
dupe_hash_ids = set()
for ( smaller_hash_id, larger_hash_id ) in self._c.execute( 'SELECT smaller_hash_id, larger_hash_id FROM duplicate_pairs WHERE duplicate_type = ? AND ( smaller_hash_id = ? OR larger_hash_id = ? );', ( HC.DUPLICATE_UNKNOWN, random_hash_id, random_hash_id ) ):
dupe_hash_ids.add( smaller_hash_id )
dupe_hash_ids.add( larger_hash_id )
dupe_hashes = self._GetHashes( dupe_hash_ids )
return dupe_hashes
def _CacheSimilarFilesMaintainDuplicatePairs( self, search_distance, job_key = None, stop_time = None, abandon_if_other_work_to_do = False ):
if abandon_if_other_work_to_do:
result = self._c.execute( 'SELECT 1 FROM shape_maintenance_phash_regen;' ).fetchone()
if result is not None:
return
result = self._c.execute( 'SELECT 1 FROM shape_maintenance_branch_regen;' ).fetchone()
if result is not None:
return
pub_job_key = False
job_key_pubbed = False
@ -1784,11 +1825,13 @@ class DB( HydrusDB.HydrusDB ):
pub_job_key = True
( total_num_hash_ids_in_cache, ) = self._c.execute( 'SELECT COUNT( * ) FROM shape_search_cache;' ).fetchone()
hash_ids = [ hash_id for ( hash_id, ) in self._c.execute( 'SELECT hash_id FROM shape_search_cache WHERE searched_distance IS NULL or searched_distance < ?;', ( search_distance, ) ) ]
pairs_found = 0
num_to_do = len( hash_ids )
total_done_previously = total_num_hash_ids_in_cache - len( hash_ids )
for ( i, hash_id ) in enumerate( hash_ids ):
@ -1810,10 +1853,11 @@ class DB( HydrusDB.HydrusDB ):
return
text = 'searched ' + HydrusData.ConvertValueRangeToPrettyString( i, num_to_do ) + ', found ' + HydrusData.ConvertIntToPrettyString( pairs_found )
text = 'searched ' + HydrusData.ConvertValueRangeToPrettyString( total_done_previously + i, total_num_hash_ids_in_cache ) + ' files, found ' + HydrusData.ConvertIntToPrettyString( pairs_found ) + ' potential duplicate pairs'
HydrusGlobals.client_controller.pub( 'splash_set_status_text', text )
job_key.SetVariable( 'popup_text_1', text )
job_key.SetVariable( 'popup_gauge_1', ( i, num_to_do ) )
job_key.SetVariable( 'popup_gauge_1', ( total_done_previously + i, total_num_hash_ids_in_cache ) )
duplicate_hash_ids = [ duplicate_hash_id for duplicate_hash_id in self._CacheSimilarFilesSearch( hash_id, search_distance ) if duplicate_hash_id != hash_id ]
@ -1843,11 +1887,13 @@ class DB( HydrusDB.HydrusDB ):
pub_job_key = True
( total_num_hash_ids_in_cache, ) = self._c.execute( 'SELECT COUNT( * ) FROM shape_search_cache;' ).fetchone()
hash_ids = [ hash_id for ( hash_id, ) in self._c.execute( 'SELECT hash_id FROM shape_maintenance_phash_regen;' ) ]
client_files_manager = self._controller.GetClientFilesManager()
num_to_do = len( hash_ids )
total_done_previously = total_num_hash_ids_in_cache - len( hash_ids )
for ( i, hash_id ) in enumerate( hash_ids ):
@ -1869,10 +1915,16 @@ class DB( HydrusDB.HydrusDB ):
return
text = 'regenerating similar file metadata - ' + HydrusData.ConvertValueRangeToPrettyString( i, num_to_do )
if i % 50 == 0:
gc.collect()
text = 'regenerating similar file metadata - ' + HydrusData.ConvertValueRangeToPrettyString( total_done_previously + i, total_num_hash_ids_in_cache )
HydrusGlobals.client_controller.pub( 'splash_set_status_text', text )
job_key.SetVariable( 'popup_text_1', text )
job_key.SetVariable( 'popup_gauge_1', ( i, num_to_do ) )
job_key.SetVariable( 'popup_gauge_1', ( total_done_previously + i, total_num_hash_ids_in_cache ) )
try:
@ -1930,7 +1982,17 @@ class DB( HydrusDB.HydrusDB ):
job_key.Delete( 30 )
def _CacheSimilarFilesMaintainTree( self, job_key = None, stop_time = None ):
def _CacheSimilarFilesMaintainTree( self, job_key = None, stop_time = None, abandon_if_other_work_to_do = False ):
if abandon_if_other_work_to_do:
result = self._c.execute( 'SELECT 1 FROM shape_maintenance_phash_regen;' ).fetchone()
if result is not None:
return
pub_job_key = False
job_key_pubbed = False
@ -1994,8 +2056,24 @@ class DB( HydrusDB.HydrusDB ):
def _CacheSimilarFilesMaintenanceDue( self ):
result = self._c.execute( 'SELECT 1 FROM shape_maintenance_phash_regen;' ).fetchone()
if result is not None:
return True
result = self._c.execute( 'SELECT 1 FROM shape_maintenance_branch_regen;' ).fetchone()
if result is not None:
return True
search_distance = HydrusGlobals.client_controller.GetNewOptions().GetInteger( 'similar_files_duplicate_pairs_search_distance' )
result = self._c.execute( 'SELECT 1 FROM shape_search_cache WHERE searched_distance IS NULL or searched_distance < ?;', ( search_distance, ) ).fetchone()
if result is not None:
return True
@ -2038,7 +2116,7 @@ class DB( HydrusDB.HydrusDB ):
for ( v_id, v_phash ) in viewpoints:
views = [ HydrusData.GetHammingDistance( v_phash, s_phash ) for ( s_id, s_phash ) in sample if v_id != s_id ]
views = [ HydrusData.Get64BitHammingDistance( v_phash, s_phash ) for ( s_id, s_phash ) in sample if v_id != s_id ]
views.sort()
@ -2202,99 +2280,112 @@ class DB( HydrusDB.HydrusDB ):
def _CacheSimilarFilesSearch( self, hash_id, max_hamming_distance ):
search_radius = max_hamming_distance
result = self._c.execute( 'SELECT phash_id FROM shape_vptree WHERE parent_id IS NULL;' ).fetchone()
if result is None:
if max_hamming_distance == 0:
return []
similar_hash_ids = [ hash_id for ( hash_id, ) in self._c.execute( 'SELECT hash_id FROM shape_perceptual_hash_map WHERE phash_id IN ( SELECT phash_id FROM shape_perceptual_hash_map WHERE hash_id = ? );', ( hash_id, ) ) ]
( root_node_phash_id, ) = result
search_phashes = [ phash for ( phash, ) in self._c.execute( 'SELECT phash FROM shape_perceptual_hashes, shape_perceptual_hash_map USING ( phash_id ) WHERE hash_id = ?;', ( hash_id, ) ) ]
if len( search_phashes ) == 0:
else:
return []
search_radius = max_hamming_distance
potentials = [ ( root_node_phash_id, tuple( search_phashes ) ) ]
similar_phash_ids = set()
num_cycles = 0
while len( potentials ) > 0:
result = self._c.execute( 'SELECT phash_id FROM shape_vptree WHERE parent_id IS NULL;' ).fetchone()
num_cycles += 1
( node_phash_id, search_phashes ) = potentials.pop( 0 )
( node_phash, node_radius, inner_phash_id, outer_phash_id ) = self._c.execute( 'SELECT phash, radius, inner_id, outer_id FROM shape_perceptual_hashes, shape_vptree USING ( phash_id ) WHERE phash_id = ?;', ( node_phash_id, ) ).fetchone()
inner_search_phashes = []
outer_search_phashes = []
for search_phash in search_phashes:
if result is None:
# first check the node--is it similar?
return []
node_hamming_distance = HydrusData.GetHammingDistance( search_phash, node_phash )
( root_node_phash_id, ) = result
search_phashes = [ phash for ( phash, ) in self._c.execute( 'SELECT phash FROM shape_perceptual_hashes, shape_perceptual_hash_map USING ( phash_id ) WHERE hash_id = ?;', ( hash_id, ) ) ]
if len( search_phashes ) == 0:
if node_hamming_distance <= search_radius:
similar_phash_ids.add( node_phash_id )
return []
# now how about its children?
next_potentials = { root_node_phash_id : tuple( search_phashes ) }
similar_phash_ids = set()
num_cycles = 0
while len( next_potentials ) > 0:
if node_radius is not None:
current_potentials = next_potentials
next_potentials = {}
num_cycles += 1
select_statement = 'SELECT phash_id, phash, radius, inner_id, outer_id FROM shape_perceptual_hashes, shape_vptree USING ( phash_id ) WHERE phash_id IN %s;'
for ( node_phash_id, node_phash, node_radius, inner_phash_id, outer_phash_id ) in self._SelectFromList( select_statement, current_potentials.keys() ):
# we have two spheres--node and search--their centers separated by node_hamming_distance
# we want to search inside/outside the node_sphere if the search_sphere intersects with those spaces
# there are four possibles:
# (----N----)-(--S--) intersects with outer only - distance between N and S > their radii
# (----N---(-)-S--) intersects with both
# (----N-(--S-)-) intersects with both
# (---(-N-S--)-) intersects with inner only - distance between N and S + radius_S does not exceed radius_N
search_phashes = current_potentials[ node_phash_id ]
spheres_disjoint = node_hamming_distance > node_radius + search_radius
search_sphere_subset_of_node_sphere = node_hamming_distance + search_radius <= node_radius
inner_search_phashes = []
outer_search_phashes = []
if not spheres_disjoint: # i.e. they intersect at some point
for search_phash in search_phashes:
inner_search_phashes.append( search_phash )
# first check the node--is it similar?
node_hamming_distance = HydrusData.Get64BitHammingDistance( search_phash, node_phash )
if node_hamming_distance <= search_radius:
similar_phash_ids.add( node_phash_id )
# now how about its children?
if node_radius is not None:
# we have two spheres--node and search--their centers separated by node_hamming_distance
# we want to search inside/outside the node_sphere if the search_sphere intersects with those spaces
# there are four possibles:
# (----N----)-(--S--) intersects with outer only - distance between N and S > their radii
# (----N---(-)-S--) intersects with both
# (----N-(--S-)-) intersects with both
# (---(-N-S--)-) intersects with inner only - distance between N and S + radius_S does not exceed radius_N
spheres_disjoint = node_hamming_distance > ( node_radius + search_radius )
search_sphere_subset_of_node_sphere = ( node_hamming_distance + search_radius ) <= node_radius
if not spheres_disjoint: # i.e. they intersect at some point
inner_search_phashes.append( search_phash )
if not search_sphere_subset_of_node_sphere: # i.e. search sphere intersects with non-node sphere space at some point
outer_search_phashes.append( search_phash )
if not search_sphere_subset_of_node_sphere: # i.e. search sphere intersects with non-node sphere space at some point
if inner_phash_id is not None and len( inner_search_phashes ) > 0:
outer_search_phashes.append( search_phash )
next_potentials[ inner_phash_id ] = tuple( inner_search_phashes )
if outer_phash_id is not None and len( outer_search_phashes ) > 0:
next_potentials[ outer_phash_id ] = tuple( outer_search_phashes )
if inner_phash_id is not None and len( inner_search_phashes ) > 0:
if HydrusGlobals.db_report_mode:
potentials.append( ( inner_phash_id, tuple( inner_search_phashes ) ) )
HydrusData.ShowText( 'Similar file search completed in ' + HydrusData.ConvertIntToPrettyString( num_cycles ) + ' cycles.' )
if outer_phash_id is not None and len( outer_search_phashes ) > 0:
with HydrusDB.TemporaryIntegerTable( self._c, similar_phash_ids, 'phash_id' ) as temp_table_name:
potentials.append( ( outer_phash_id, tuple( outer_search_phashes ) ) )
similar_hash_ids = [ hash_id for ( hash_id, ) in self._c.execute( 'SELECT hash_id FROM shape_perceptual_hash_map, ' + temp_table_name + ' USING ( phash_id );' ) ]
if HydrusGlobals.db_report_mode:
HydrusData.ShowText( 'Similar file search completed in ' + HydrusData.ConvertIntToPrettyString( num_cycles ) + ' cycles.' )
with HydrusDB.TemporaryIntegerTable( self._c, similar_phash_ids, 'phash_id' ) as temp_table_name:
similar_hash_ids = [ hash_id for ( hash_id, ) in self._c.execute( 'SELECT hash_id FROM shape_perceptual_hash_map, ' + temp_table_name + ' USING ( phash_id );' ) ]
return similar_hash_ids
@ -7295,6 +7386,7 @@ class DB( HydrusDB.HydrusDB ):
elif action == 'service_info': result = self._GetServiceInfo( *args, **kwargs )
elif action == 'services': result = self._GetServices( *args, **kwargs )
elif action == 'similar_files_maintenance_status': result = self._CacheSimilarFilesGetMaintenanceStatus( *args, **kwargs )
elif action == 'some_dupes': result = self._CacheSimilarFilesGetSomeDupes( *args, **kwargs )
elif action == 'related_tags': result = self._GetRelatedTags( *args, **kwargs )
elif action == 'tag_censorship': result = self._GetTagCensorship( *args, **kwargs )
elif action == 'tag_parents': result = self._GetTagParents( *args, **kwargs )
@ -8839,7 +8931,7 @@ class DB( HydrusDB.HydrusDB ):
else:
children = [ ( HydrusData.GetHammingDistance( phash, child_phash ), child_id, child_phash ) for ( child_id, child_phash ) in children ]
children = [ ( HydrusData.Get64BitHammingDistance( phash, child_phash ), child_id, child_phash ) for ( child_id, child_phash ) in children ]
children.sort()
@ -8973,6 +9065,34 @@ class DB( HydrusDB.HydrusDB ):
self._c.execute( 'DELETE FROM web_sessions WHERE name = ?;', ( 'pixiv', ) )
if version == 241:
try:
subscriptions = self._GetJSONDumpNamed( HydrusSerialisable.SERIALISABLE_TYPE_SUBSCRIPTION )
for subscription in subscriptions:
g_i = subscription._gallery_identifier
if g_i.GetSiteType() == HC.SITE_TYPE_BOORU:
if 'sankaku' in g_i.GetAdditionalInfo():
subscription._paused = True
self._SetJSONDump( subscription )
except Exception as e:
HydrusData.Print( 'While attempting to pause all sankaku subs, I had this problem:' )
HydrusData.PrintException( e )
self._controller.pub( 'splash_set_title_text', 'updated db to v' + str( version + 1 ) )
self._c.execute( 'UPDATE version SET version = ?;', ( version + 1, ) )

57
include/ClientData.py
View File

@ -558,10 +558,14 @@ class ClientOptions( HydrusSerialisable.SerialisableBase ):
self._dictionary[ 'booleans' ][ 'hide_message_manager_on_gui_iconise' ] = HC.PLATFORM_OSX
self._dictionary[ 'booleans' ][ 'hide_message_manager_on_gui_deactive' ] = False
self._dictionary[ 'booleans' ][ 'load_images_with_pil' ] = True
self._dictionary[ 'booleans' ][ 'load_images_with_pil' ] = HC.PLATFORM_LINUX or HC.PLATFORM_OSX
self._dictionary[ 'booleans' ][ 'use_system_ffmpeg' ] = False
self._dictionary[ 'booleans' ][ 'maintain_similar_files_phashes_during_idle' ] = False
self._dictionary[ 'booleans' ][ 'maintain_similar_files_tree_during_idle' ] = False
self._dictionary[ 'booleans' ][ 'maintain_similar_files_duplicate_pairs_during_idle' ] = False
#
self._dictionary[ 'integers' ] = {}
@ -1071,6 +1075,14 @@ class ClientOptions( HydrusSerialisable.SerialisableBase ):
def InvertBoolean( self, name ):
with self._lock:
self._dictionary[ 'booleans' ][ name ] = not self._dictionary[ 'booleans' ][ name ]
def SetBoolean( self, name, value ):
with self._lock:
@ -1184,45 +1196,6 @@ class ClientOptions( HydrusSerialisable.SerialisableBase ):
HydrusSerialisable.SERIALISABLE_TYPES_TO_OBJECT_TYPES[ HydrusSerialisable.SERIALISABLE_TYPE_CLIENT_OPTIONS ] = ClientOptions
class CommandHandler( object ):
def __init__( self, action_cache ):
self._action_cache = action_cache
self._commands_to_callables = {}
def ProcessMenuEvent( self, menu_event ):
event_id = menu_event.GetId()
action = self._action_cache.GetAction( event_id )
if action is not None:
( command, data ) = action
if command in self._commands_to_callables:
callable = self._commands_to_callables[ command ]
if data is None:
wx.CallAfter( callable )
else:
wx.CallAfter( callable, data )
def SetCallable( self, command, callable ):
self._commands_to_callables[ command ] = callable
class Credentials( HydrusData.HydrusYAMLBase ):
yaml_tag = u'!Credentials'
@ -1773,7 +1746,7 @@ class ServiceRestricted( ServiceRemote ):
( host, port ) = credentials.GetAddress()
url = 'http://' + host + ':' + str( port ) + path_and_query
url = 'https://' + host + ':' + str( port ) + path_and_query
( response, size_of_response, response_headers, cookies ) = HydrusGlobals.client_controller.DoHTTP( method, url, request_headers, body, report_hooks = report_hooks, temp_path = temp_path, hydrus_network = True )
@ -2341,6 +2314,8 @@ class ServiceRepository( ServiceRestricted ):
HydrusData.Print( job_key.ToString() )
job_key.Finish()
time.sleep( 3 )
job_key.Delete()

9
include/ClientDefaults.py
View File

@ -131,6 +131,7 @@ def GetClientDefaultOptions():
shortcuts[ wx.ACCEL_CTRL ][ ord( 'B' ) ] = 'frame_back'
shortcuts[ wx.ACCEL_CTRL ][ ord( 'N' ) ] = 'frame_next'
shortcuts[ wx.ACCEL_CTRL ][ ord( 'T' ) ] = 'new_page'
shortcuts[ wx.ACCEL_CTRL ][ ord( 'U' ) ] = 'unclose_page'
shortcuts[ wx.ACCEL_CTRL ][ ord( 'W' ) ] = 'close_page'
shortcuts[ wx.ACCEL_CTRL ][ ord( 'R' ) ] = 'show_hide_splitters'
shortcuts[ wx.ACCEL_CTRL ][ ord( 'S' ) ] = 'set_search_focus'
@ -265,7 +266,7 @@ def GetDefaultNamespacesAndSearchValue( gallery_identifier ):
elif site_type in ( HC.SITE_TYPE_HENTAI_FOUNDRY, HC.SITE_TYPE_HENTAI_FOUNDRY_ARTIST, HC.SITE_TYPE_HENTAI_FOUNDRY_TAGS ):
namespaces = [ 'creator', 'title', '' ]
namespaces = [ 'creator', 'title' ]
if site_type == HC.SITE_TYPE_HENTAI_FOUNDRY:
@ -367,7 +368,7 @@ def GetDefaultBoorus():
image_data = None
tag_classnames_to_namespaces = { 'category-0' : '', 'category-4' : 'character', 'category-3' : 'series', 'category-1' : 'creator' }
boorus[ 'danbooru' ] = ClientData.Booru( name, search_url, search_separator, advance_by_page_num, thumb_classname, image_id, image_data, tag_classnames_to_namespaces )
#boorus[ 'danbooru' ] = ClientData.Booru( name, search_url, search_separator, advance_by_page_num, thumb_classname, image_id, image_data, tag_classnames_to_namespaces )
name = 'mishimmie'
search_url = 'http://shimmie.katawa-shoujo.com/post/list/%tags%/%index%'
@ -455,7 +456,7 @@ def GetDefaultBoorus():
image_data = None
tag_classnames_to_namespaces = { 'tag-type-general' : '', 'tag-type-character' : 'character', 'tag-type-copyright' : 'series', 'tag-type-artist' : 'creator', 'tag-type-medium' : 'medium' }
boorus[ 'sankaku chan' ] = ClientData.Booru( name, search_url, search_separator, advance_by_page_num, thumb_classname, image_id, image_data, tag_classnames_to_namespaces )
#boorus[ 'sankaku chan' ] = ClientData.Booru( name, search_url, search_separator, advance_by_page_num, thumb_classname, image_id, image_data, tag_classnames_to_namespaces )
name = 'rule34hentai'
search_url = 'http://rule34hentai.net/post/list/%tags%/%index%'
@ -572,4 +573,4 @@ def GetDefaultScriptRows():
script_info.append( ( 32, 'iqdb danbooru', 1, '''["http://danbooru.iqdb.org/", 1, 0, 0, "file", {}, [[29, 1, ["link to danbooru", [27, 1, [[["td", {"class": "image"}, 1], ["a", {}, 0]], "href"]], [[30, 1, ["", 0, [27, 1, [[["section", {"id": "tag-list"}, 0], ["li", {"class": "category-1"}, null], ["a", {"class": "search-tag"}, 0]], null]], "creator"]], [30, 1, ["", 0, [27, 1, [[["section", {"id": "tag-list"}, 0], ["li", {"class": "category-3"}, null], ["a", {"class": "search-tag"}, 0]], null]], "series"]], [30, 1, ["", 0, [27, 1, [[["section", {"id": "tag-list"}, 0], ["li", {"class": "category-4"}, null], ["a", {"class": "search-tag"}, 0]], null]], "character"]], [30, 1, ["", 0, [27, 1, [[["section", {"id": "tag-list"}, 0], ["li", {"class": "category-0"}, null], ["a", {"class": "search-tag"}, 0]], null]], ""]]]]], [30, 1, ["no iqdb match found", 8, [27, 1, [[["th", {}, null]], null]], [false, true, "Best match"]]]]]''' ) )
return script_info

141
include/ClientDownloading.py
View File

@ -233,7 +233,8 @@ def GetYoutubeFormats( youtube_url ):
def THREADDownloadURL( job_key, url, url_string ):
job_key.SetVariable( 'popup_text_1', url_string + ' - initialising' )
job_key.SetVariable( 'popup_title', url_string )
job_key.SetVariable( 'popup_text_1', 'initialising' )
( os_file_handle, temp_path ) = HydrusPaths.GetTempPath()
@ -241,47 +242,21 @@ def THREADDownloadURL( job_key, url, url_string ):
response = ClientNetworking.RequestsGet( url, stream = True )
if 'content-length' in response.headers:
gauge_range = int( response.headers[ 'content-length' ] )
else:
gauge_range = None
gauge_value = 0
with open( temp_path, 'wb' ) as f:
for chunk in response.iter_content( chunk_size = 65536 ):
( i_paused, should_quit ) = job_key.WaitIfNeeded()
if should_quit:
return
f.write( chunk )
gauge_value += len( chunk )
if gauge_range is None: text = url_string + ' - ' + HydrusData.ConvertIntToBytes( gauge_value )
else: text = url_string + ' - ' + HydrusData.ConvertValueRangeToBytes( gauge_value, gauge_range )
job_key.SetVariable( 'popup_text_1', text )
job_key.SetVariable( 'popup_gauge_1', ( gauge_value, gauge_range ) )
ClientNetworking.StreamResponseToFile( job_key, response, f )
job_key.DeleteVariable( 'popup_gauge_1' )
job_key.SetVariable( 'popup_text_1', 'importing ' + url_string )
job_key.SetVariable( 'popup_text_1', 'importing' )
client_files_manager = HydrusGlobals.client_controller.GetClientFilesManager()
( result, hash ) = client_files_manager.ImportFile( temp_path )
except HydrusExceptions.CancelledException:
return
except HydrusExceptions.NetworkException:
job_key.Cancel()
@ -297,25 +272,26 @@ def THREADDownloadURL( job_key, url, url_string ):
if result == CC.STATUS_SUCCESSFUL:
job_key.SetVariable( 'popup_text_1', url_string )
job_key.SetVariable( 'popup_text_1', 'successful!' )
else:
job_key.SetVariable( 'popup_text_1', url_string + ' was already in the database!' )
job_key.SetVariable( 'popup_text_1', 'was already in the database!' )
job_key.SetVariable( 'popup_files', { hash } )
elif result == CC.STATUS_DELETED:
job_key.SetVariable( 'popup_text_1', url_string + ' had already been deleted!' )
job_key.SetVariable( 'popup_text_1', 'had already been deleted!' )
job_key.Finish()
def THREADDownloadURLs( job_key, urls, title ):
job_key.SetVariable( 'popup_text_1', title + ' - initialising' )
job_key.SetVariable( 'popup_title', title )
job_key.SetVariable( 'popup_text_1', 'initialising' )
num_successful = 0
num_redundant = 0
@ -333,69 +309,52 @@ def THREADDownloadURLs( job_key, urls, title ):
break
job_key.SetVariable( 'popup_text_1', title + ' - ' + HydrusData.ConvertValueRangeToPrettyString( i + 1, len( urls ) ) )
job_key.SetVariable( 'popup_text_1', HydrusData.ConvertValueRangeToPrettyString( i + 1, len( urls ) ) )
job_key.SetVariable( 'popup_gauge_1', ( i + 1, len( urls ) ) )
( os_file_handle, temp_path ) = HydrusPaths.GetTempPath()
try:
response = ClientNetworking.RequestsGet( url, stream = True )
if 'content-length' in response.headers:
try:
gauge_range = int( response.headers[ 'content-length' ] )
response = ClientNetworking.RequestsGet( url, stream = True )
else:
gauge_range = None
gauge_value = 0
with open( temp_path, 'wb' ) as f:
for chunk in response.iter_content( chunk_size = 65536 ):
with open( temp_path, 'wb' ) as f:
( i_paused, should_quit ) = job_key.WaitIfNeeded()
if should_quit:
return
f.write( chunk )
gauge_value += len( chunk )
if gauge_range is None: text = 'downloading - ' + HydrusData.ConvertIntToBytes( gauge_value )
else: text = 'downloading - ' + HydrusData.ConvertValueRangeToBytes( gauge_value, gauge_range )
job_key.SetVariable( 'popup_text_2', text )
job_key.SetVariable( 'popup_gauge_2', ( gauge_value, gauge_range ) )
ClientNetworking.StreamResponseToFile( job_key, response, f )
except HydrusExceptions.CancelledException:
return
except HydrusExceptions.NetworkException:
job_key.Cancel()
raise
job_key.SetVariable( 'popup_text_2', 'importing' )
client_files_manager = HydrusGlobals.client_controller.GetClientFilesManager()
( result, hash ) = client_files_manager.ImportFile( temp_path )
except HydrusExceptions.NetworkException:
job_key.Cancel()
raise
except Exception as e:
HydrusData.Print( url + ' failed to import!' )
HydrusData.PrintException( e )
num_failed += 1
continue
try:
job_key.SetVariable( 'popup_text_2', 'importing' )
client_files_manager = HydrusGlobals.client_controller.GetClientFilesManager()
( result, hash ) = client_files_manager.ImportFile( temp_path )
except Exception as e:
job_key.DeleteVariable( 'popup_text_2' )
HydrusData.Print( url + ' failed to import!' )
HydrusData.PrintException( e )
num_failed += 1
continue
finally:
@ -443,7 +402,7 @@ def THREADDownloadURLs( job_key, urls, title ):
text_components.append( HydrusData.ConvertIntToPrettyString( num_failed ) + ' failed (errors written to log)' )
job_key.SetVariable( 'popup_text_1', title + ' - ' + ', '.join( text_components ) )
job_key.SetVariable( 'popup_text_1', ', '.join( text_components ) )
if len( successful_hashes ) > 0:
@ -1404,10 +1363,6 @@ class GalleryHentaiFoundry( Gallery ):
except: pass
tag_links = soup.find_all( 'a', rel = 'tag' )
for tag_link in tag_links: tags.append( tag_link.string )
return ( image_url, tags )

188
include/ClientGUI.py
View File

@ -757,13 +757,15 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
elif isinstance( o, types.BuiltinMethodType ): class_count[ o.__name__ ] += 1
HydrusData.Print( 'gc:' )
HydrusData.Print( 'gc types:' )
for ( k, v ) in count.items():
if v > 100: print ( k, v )
HydrusData.Print( 'gc classes:' )
for ( k, v ) in class_count.items():
if v > 100: print ( k, v )
@ -865,24 +867,24 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
def file():
ClientGUIMenus.AppendMenuItem( menu, 'import files', 'Add new files to the database.', self, self._ImportFiles )
ClientGUIMenus.AppendMenuItem( self, menu, 'import files', 'Add new files to the database.', self._ImportFiles )
menu.AppendSeparator()
ClientGUIMenus.AppendMenuItem( menu, 'manage import folders', 'Manage folders from which the client can automatically import.', self, self._ManageImportFolders )
ClientGUIMenus.AppendMenuItem( menu, 'manage export folders', 'Manage folders to which the client can automatically export.', self, self._ManageExportFolders )
ClientGUIMenus.AppendMenuItem( self, menu, 'manage import folders', 'Manage folders from which the client can automatically import.', self._ManageImportFolders )
ClientGUIMenus.AppendMenuItem( self, menu, 'manage export folders', 'Manage folders to which the client can automatically export.', self._ManageExportFolders )
menu.AppendSeparator()
open = wx.Menu()
ClientGUIMenus.AppendMenuItem( open, 'installation directory', 'Open the installation directory for this client.', self, self._OpenInstallFolder )
ClientGUIMenus.AppendMenuItem( open, 'database directory', 'Open the database directory for this instance of the client.', self, self._OpenDBFolder )
ClientGUIMenus.AppendMenuItem( open, 'quick export directory', 'Open the export directory so you can easily access the files you have exported.', self, self._OpenExportFolder )
ClientGUIMenus.AppendMenuItem( self, open, 'installation directory', 'Open the installation directory for this client.', self._OpenInstallFolder )
ClientGUIMenus.AppendMenuItem( self, open, 'database directory', 'Open the database directory for this instance of the client.', self._OpenDBFolder )
ClientGUIMenus.AppendMenuItem( self, open, 'quick export directory', 'Open the export directory so you can easily access the files you have exported.', self._OpenExportFolder )
ClientGUIMenus.AppendMenu( menu, open, 'open' )
menu.AppendSeparator()
ClientGUIMenus.AppendMenuItem( menu, 'options', 'Change how the client operates.', self, self._ManageOptions )
ClientGUIMenus.AppendMenuItem( self, menu, 'options', 'Change how the client operates.', self._ManageOptions )
menu.AppendSeparator()
@ -890,10 +892,10 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
if not we_borked_linux_pyinstaller:
ClientGUIMenus.AppendMenuItem( menu, 'restart', 'Shut the client down and then start it up again.', self, self.Exit, restart = True )
ClientGUIMenus.AppendMenuItem( self, menu, 'restart', 'Shut the client down and then start it up again.', self.Exit, restart = True )
ClientGUIMenus.AppendMenuItem( menu, 'exit', 'Shut the client down.', self, self.Exit )
ClientGUIMenus.AppendMenuItem( self, menu, 'exit', 'Shut the client down.', self.Exit )
return ( menu, p( '&File' ), True )
@ -921,14 +923,14 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
did_undo_stuff = True
ClientGUIMenus.AppendMenuItem( menu, undo_string, 'Undo last operation.', self, self._controller.pub, 'undo' )
ClientGUIMenus.AppendMenuItem( self, menu, undo_string, 'Undo last operation.', self._controller.pub, 'undo' )
if redo_string is not None:
did_undo_stuff = True
ClientGUIMenus.AppendMenuItem( menu, redo_string, 'Redo last operation.', self, self._controller.pub, 'redo' )
ClientGUIMenus.AppendMenuItem( self, menu, redo_string, 'Redo last operation.', self._controller.pub, 'redo' )
if have_closed_pages:
@ -940,7 +942,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
undo_pages = wx.Menu()
ClientGUIMenus.AppendMenuItem( undo_pages, 'clear all', 'Remove all closed pages from memory.', self, self._DeleteAllClosedPages )
ClientGUIMenus.AppendMenuItem( self, undo_pages, 'clear all', 'Remove all closed pages from memory.', self._DeleteAllClosedPages )
undo_pages.AppendSeparator()
@ -958,7 +960,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
for ( index, name ) in args:
ClientGUIMenus.AppendMenuItem( undo_pages, name, 'Restore this page.', self, self._UnclosePage, index )
ClientGUIMenus.AppendMenuItem( self, undo_pages, name, 'Restore this page.', self._UnclosePage, index )
ClientGUIMenus.AppendMenu( menu, undo_pages, 'closed pages' )
@ -974,8 +976,8 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
def pages():
ClientGUIMenus.AppendMenuItem( menu, 'refresh', 'If the current page has a search, refresh it.', self, self._Refresh )
ClientGUIMenus.AppendMenuItem( menu, 'show/hide management and preview panels', 'Show or hide the panels on the left.', self, self._ShowHideSplitters )
ClientGUIMenus.AppendMenuItem( self, menu, 'refresh', 'If the current page has a search, refresh it.', self._Refresh )
ClientGUIMenus.AppendMenuItem( self, menu, 'show/hide management and preview panels', 'Show or hide the panels on the left.', self._ShowHideSplitters )
menu.AppendSeparator()
@ -989,13 +991,13 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
for name in gui_session_names:
ClientGUIMenus.AppendMenuItem( load, name, 'Close all other pages and load this session.', self, self._LoadGUISession, name )
ClientGUIMenus.AppendMenuItem( self, load, name, 'Close all other pages and load this session.', self._LoadGUISession, name )
ClientGUIMenus.AppendMenu( sessions, load, 'load' )
ClientGUIMenus.AppendMenuItem( sessions, 'save current', 'Save the existing open pages as a session.', self, self._SaveGUISession )
ClientGUIMenus.AppendMenuItem( self, sessions, 'save current', 'Save the existing open pages as a session.', self._SaveGUISession )
if len( gui_session_names ) > 0 and gui_session_names != [ 'last session' ]:
@ -1005,7 +1007,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
if name != 'last session':
ClientGUIMenus.AppendMenuItem( delete, name, 'Delete this session.', self, self._DeleteGUISession, name )
ClientGUIMenus.AppendMenuItem( self, delete, name, 'Delete this session.', self._DeleteGUISession, name )
@ -1016,7 +1018,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
menu.AppendSeparator()
ClientGUIMenus.AppendMenuItem( menu, 'pick a new page', 'Choose a new page to open.', self, self._ChooseNewPage )
ClientGUIMenus.AppendMenuItem( self, menu, 'pick a new page', 'Choose a new page to open.', self._ChooseNewPage )
#
@ -1032,17 +1034,17 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
petition_resolve_file_services = [ repository for repository in file_repositories if repository.GetInfo( 'account' ).HasPermission( HC.RESOLVE_PETITIONS ) ]
ClientGUIMenus.AppendMenuItem( search_menu, 'my files', 'Open a new search tab for your files.', self, self._NewPageQuery, CC.LOCAL_FILE_SERVICE_KEY )
ClientGUIMenus.AppendMenuItem( search_menu, 'trash', 'Open a new search tab for your recently deleted files.', self, self._NewPageQuery, CC.TRASH_SERVICE_KEY )
ClientGUIMenus.AppendMenuItem( self, search_menu, 'my files', 'Open a new search tab for your files.', self._NewPageQuery, CC.LOCAL_FILE_SERVICE_KEY )
ClientGUIMenus.AppendMenuItem( self, search_menu, 'trash', 'Open a new search tab for your recently deleted files.', self._NewPageQuery, CC.TRASH_SERVICE_KEY )
for service in file_repositories:
ClientGUIMenus.AppendMenuItem( search_menu, service.GetName(), 'Open a new search tab for ' + service.GetName() + '.', self, self._NewPageQuery, service.GetServiceKey() )
ClientGUIMenus.AppendMenuItem( self, search_menu, service.GetName(), 'Open a new search tab for ' + service.GetName() + '.', self._NewPageQuery, service.GetServiceKey() )
search_menu.AppendSeparator()
ClientGUIMenus.AppendMenuItem( search_menu, 'duplicates (under construction!)', 'Open a new tab to discover and filter duplicate files.', self, self._NewPageDuplicateFilter )
ClientGUIMenus.AppendMenuItem( self, search_menu, 'duplicates (under construction!)', 'Open a new tab to discover and filter duplicate files.', self._NewPageDuplicateFilter )
ClientGUIMenus.AppendMenu( menu, search_menu, 'new search page' )
@ -1054,12 +1056,12 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
for service in petition_resolve_tag_services:
ClientGUIMenus.AppendMenuItem( petition_menu, service.GetName(), 'Open a new tag petition tab for ' + service.GetName() + '.', self, self._NewPagePetitions, service.GetServiceKey() )
ClientGUIMenus.AppendMenuItem( self, petition_menu, service.GetName(), 'Open a new tag petition tab for ' + service.GetName() + '.', self._NewPagePetitions, service.GetServiceKey() )
for service in petition_resolve_file_services:
ClientGUIMenus.AppendMenuItem( petition_menu, service.GetName(), 'Open a new file petition tab for ' + service.GetName() + '.', self, self._NewPagePetitions, service.GetServiceKey() )
ClientGUIMenus.AppendMenuItem( self, petition_menu, service.GetName(), 'Open a new file petition tab for ' + service.GetName() + '.', self._NewPagePetitions, service.GetServiceKey() )
ClientGUIMenus.AppendMenu( menu, petition_menu, 'new petition page' )
@ -1069,23 +1071,23 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
download_menu = wx.Menu()
ClientGUIMenus.AppendMenuItem( download_menu, 'url download', 'Open a new tab to download some raw urls.', self, self._NewPageImportURLs )
ClientGUIMenus.AppendMenuItem( download_menu, 'thread watcher', 'Open a new tab to watch a thread.', self, self._NewPageImportThreadWatcher )
ClientGUIMenus.AppendMenuItem( download_menu, 'webpage of images', 'Open a new tab to download files from generic galleries or threads.', self, self._NewPageImportPageOfImages )
ClientGUIMenus.AppendMenuItem( self, download_menu, 'url download', 'Open a new tab to download some raw urls.', self._NewPageImportURLs )
ClientGUIMenus.AppendMenuItem( self, download_menu, 'thread watcher', 'Open a new tab to watch a thread.', self._NewPageImportThreadWatcher )
ClientGUIMenus.AppendMenuItem( self, download_menu, 'webpage of images', 'Open a new tab to download files from generic galleries or threads.', self._NewPageImportPageOfImages )
gallery_menu = wx.Menu()
ClientGUIMenus.AppendMenuItem( gallery_menu, 'booru', 'Open a new tab to download files from a booru.', self, self._NewPageImportBooru )
ClientGUIMenus.AppendMenuItem( gallery_menu, 'deviant art', 'Open a new tab to download files from Deviant Art.', self, self._NewPageImportGallery, ClientDownloading.GalleryIdentifier( HC.SITE_TYPE_DEVIANT_ART ) )
ClientGUIMenus.AppendMenuItem( self, gallery_menu, 'booru', 'Open a new tab to download files from a booru.', self._NewPageImportBooru )
ClientGUIMenus.AppendMenuItem( self, gallery_menu, 'deviant art', 'Open a new tab to download files from Deviant Art.', self._NewPageImportGallery, ClientDownloading.GalleryIdentifier( HC.SITE_TYPE_DEVIANT_ART ) )
hf_submenu = wx.Menu()
ClientGUIMenus.AppendMenuItem( hf_submenu, 'by artist', 'Open a new tab to download files from Hentai Foundry.', self, self._NewPageImportGallery, ClientDownloading.GalleryIdentifier( HC.SITE_TYPE_HENTAI_FOUNDRY_ARTIST ) )
ClientGUIMenus.AppendMenuItem( hf_submenu, 'by tags', 'Open a new tab to download files from Hentai Foundry.', self, self._NewPageImportGallery, ClientDownloading.GalleryIdentifier( HC.SITE_TYPE_HENTAI_FOUNDRY_TAGS ) )
ClientGUIMenus.AppendMenuItem( self, hf_submenu, 'by artist', 'Open a new tab to download files from Hentai Foundry.', self._NewPageImportGallery, ClientDownloading.GalleryIdentifier( HC.SITE_TYPE_HENTAI_FOUNDRY_ARTIST ) )
ClientGUIMenus.AppendMenuItem( self, hf_submenu, 'by tags', 'Open a new tab to download files from Hentai Foundry.', self._NewPageImportGallery, ClientDownloading.GalleryIdentifier( HC.SITE_TYPE_HENTAI_FOUNDRY_TAGS ) )
ClientGUIMenus.AppendMenu( gallery_menu, hf_submenu, 'hentai foundry' )
ClientGUIMenus.AppendMenuItem( gallery_menu, 'newgrounds', 'Open a new tab to download files from Newgrounds.', self, self._NewPageImportGallery, ClientDownloading.GalleryIdentifier( HC.SITE_TYPE_NEWGROUNDS ) )
ClientGUIMenus.AppendMenuItem( self, gallery_menu, 'newgrounds', 'Open a new tab to download files from Newgrounds.', self._NewPageImportGallery, ClientDownloading.GalleryIdentifier( HC.SITE_TYPE_NEWGROUNDS ) )
result = self._controller.Read( 'serialisable_simple', 'pixiv_account' )
@ -1093,26 +1095,26 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
pixiv_submenu = wx.Menu()
ClientGUIMenus.AppendMenuItem( pixiv_submenu, 'by artist id', 'Open a new tab to download files from Pixiv.', self, self._NewPageImportGallery, ClientDownloading.GalleryIdentifier( HC.SITE_TYPE_PIXIV_ARTIST_ID ) )
ClientGUIMenus.AppendMenuItem( pixiv_submenu, 'by tag', 'Open a new tab to download files from Pixiv.', self, self._NewPageImportGallery, ClientDownloading.GalleryIdentifier( HC.SITE_TYPE_PIXIV_TAG ) )
ClientGUIMenus.AppendMenuItem( self, pixiv_submenu, 'by artist id', 'Open a new tab to download files from Pixiv.', self._NewPageImportGallery, ClientDownloading.GalleryIdentifier( HC.SITE_TYPE_PIXIV_ARTIST_ID ) )
ClientGUIMenus.AppendMenuItem( self, pixiv_submenu, 'by tag', 'Open a new tab to download files from Pixiv.', self._NewPageImportGallery, ClientDownloading.GalleryIdentifier( HC.SITE_TYPE_PIXIV_TAG ) )
ClientGUIMenus.AppendMenu( gallery_menu, pixiv_submenu, 'pixiv' )
ClientGUIMenus.AppendMenuItem( gallery_menu, 'tumblr', 'Open a new tab to download files from tumblr.', self, self._NewPageImportGallery, ClientDownloading.GalleryIdentifier( HC.SITE_TYPE_TUMBLR ) )
ClientGUIMenus.AppendMenuItem( self, gallery_menu, 'tumblr', 'Open a new tab to download files from tumblr.', self._NewPageImportGallery, ClientDownloading.GalleryIdentifier( HC.SITE_TYPE_TUMBLR ) )
ClientGUIMenus.AppendMenu( download_menu, gallery_menu, 'gallery' )
ClientGUIMenus.AppendMenu( menu, download_menu, 'new download page' )
download_popup_menu = wx.Menu()
ClientGUIMenus.AppendMenuItem( download_popup_menu, 'a youtube video', 'Enter a YouTube URL and choose which formats you would like to download', self, self._StartYoutubeDownload )
ClientGUIMenus.AppendMenuItem( self, download_popup_menu, 'a youtube video', 'Enter a YouTube URL and choose which formats you would like to download', self._StartYoutubeDownload )
has_ipfs = len( [ service for service in services if service.GetServiceType() == HC.IPFS ] )
if has_ipfs:
ClientGUIMenus.AppendMenuItem( download_popup_menu, 'an ipfs multihash', 'Enter an IPFS multihash and attempt to import whatever is returned.', self, self._StartIPFSDownload )
ClientGUIMenus.AppendMenuItem( self, download_popup_menu, 'an ipfs multihash', 'Enter an IPFS multihash and attempt to import whatever is returned.', self._StartIPFSDownload )
ClientGUIMenus.AppendMenu( menu, download_popup_menu, 'new download popup' )
@ -1124,37 +1126,36 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
def database():
ClientGUIMenus.AppendMenuItem( menu, 'set a password', 'Set a simple password for the database so only you can open it in the client.', self, self._SetPassword )
ClientGUIMenus.AppendMenuItem( self, menu, 'set a password', 'Set a simple password for the database so only you can open it in the client.', self._SetPassword )
menu.AppendSeparator()
ClientGUIMenus.AppendMenuItem( menu, 'create a database backup', 'Back the database up to an external location.', self, self._controller.BackupDatabase )
ClientGUIMenus.AppendMenuItem( menu, 'restore a database backup', 'Restore the database from an external location.', self, self._controller.RestoreDatabase )
ClientGUIMenus.AppendMenuItem( self, menu, 'create a database backup', 'Back the database up to an external location.', self._controller.BackupDatabase )
ClientGUIMenus.AppendMenuItem( self, menu, 'restore a database backup', 'Restore the database from an external location.', self._controller.RestoreDatabase )
menu.AppendSeparator()
submenu = wx.Menu()
ClientGUIMenus.AppendMenuItem( submenu, 'vacuum', 'Defrag the database by completely rebuilding it.', self, self._VacuumDatabase )
ClientGUIMenus.AppendMenuItem( submenu, 'analyze', 'Optimise slow queries by running statistical analyses on the database.', self, self._AnalyzeDatabase )
ClientGUIMenus.AppendMenuItem( submenu, 'similar files search data', 'Rebalance and update the data the database uses to find similar files.', self, self._MaintainSimilarFilesData )
ClientGUIMenus.AppendMenuItem( submenu, 'rebalance file storage', 'Move your files around your chosen storage directories until they satisfy the weights you have set in the options.', self, self._RebalanceClientFiles )
ClientGUIMenus.AppendMenuItem( submenu, 'clear orphans', 'Clear out surplus files that have found their way into the file structure.', self, self._ClearOrphans )
ClientGUIMenus.AppendMenuItem( self, submenu, 'vacuum', 'Defrag the database by completely rebuilding it.', self._VacuumDatabase )
ClientGUIMenus.AppendMenuItem( self, submenu, 'analyze', 'Optimise slow queries by running statistical analyses on the database.', self._AnalyzeDatabase )
ClientGUIMenus.AppendMenuItem( self, submenu, 'rebalance file storage', 'Move your files around your chosen storage directories until they satisfy the weights you have set in the options.', self._RebalanceClientFiles )
ClientGUIMenus.AppendMenuItem( self, submenu, 'clear orphans', 'Clear out surplus files that have found their way into the file structure.', self._ClearOrphans )
ClientGUIMenus.AppendMenu( menu, submenu, 'maintain' )
submenu = wx.Menu()
ClientGUIMenus.AppendMenuItem( submenu, 'database integrity', 'Have the database examine all its records for internal consistency.', self, self._CheckDBIntegrity )
ClientGUIMenus.AppendMenuItem( submenu, 'file integrity', 'Have the database check if it truly has the files it thinks it does, and remove records when not.', self, self._CheckFileIntegrity )
ClientGUIMenus.AppendMenuItem( self, submenu, 'database integrity', 'Have the database examine all its records for internal consistency.', self._CheckDBIntegrity )
ClientGUIMenus.AppendMenuItem( self, submenu, 'file integrity', 'Have the database check if it truly has the files it thinks it does, and remove records when not.', self._CheckFileIntegrity )
ClientGUIMenus.AppendMenu( menu, submenu, 'check' )
submenu = wx.Menu()
ClientGUIMenus.AppendMenuItem( submenu, 'autocomplete cache', 'Delete and recreate the tag autocomplete cache, fixing any miscounts.', self, self._RegenerateACCache )
ClientGUIMenus.AppendMenuItem( submenu, 'similar files search data', 'Delete and recreate the similar files search tree.', self, self._RegenerateSimilarFilesData )
ClientGUIMenus.AppendMenuItem( submenu, 'all thumbnails', 'Delete all thumbnails and regenerate them from their original files.', self, self._RegenerateThumbnails )
ClientGUIMenus.AppendMenuItem( self, submenu, 'autocomplete cache', 'Delete and recreate the tag autocomplete cache, fixing any miscounts.', self._RegenerateACCache )
ClientGUIMenus.AppendMenuItem( self, submenu, 'similar files search data', 'Delete and recreate the similar files search tree.', self._RegenerateSimilarFilesData )
ClientGUIMenus.AppendMenuItem( self, submenu, 'all thumbnails', 'Delete all thumbnails and regenerate them from their original files.', self._RegenerateThumbnails )
ClientGUIMenus.AppendMenu( menu, submenu, 'regenerate' )
@ -1205,8 +1206,8 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
submenu = wx.Menu()
ClientGUIMenus.AppendMenuItem( submenu, 'commit', 'Upload ' + name + '\'s pending content.', self, self._UploadPending, service_key )
ClientGUIMenus.AppendMenuItem( submenu, 'forget', 'Clear ' + name + '\'s pending content.', self, self._DeletePending, service_key )
ClientGUIMenus.AppendMenuItem( self, submenu, 'commit', 'Upload ' + name + '\'s pending content.', self._UploadPending, service_key )
ClientGUIMenus.AppendMenuItem( self, submenu, 'forget', 'Clear ' + name + '\'s pending content.', self._DeletePending, service_key )
submessages = []
@ -1353,43 +1354,43 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
def help():
ClientGUIMenus.AppendMenuItem( menu, 'help', 'Open hydrus\'s local help in your web browser.', self, webbrowser.open, 'file://' + HC.HELP_DIR + '/index.html' )
ClientGUIMenus.AppendMenuItem( self, menu, 'help', 'Open hydrus\'s local help in your web browser.', webbrowser.open, 'file://' + HC.HELP_DIR + '/index.html' )
dont_know = wx.Menu()
ClientGUIMenus.AppendMenuItem( dont_know, 'just set up some repositories for me, please', 'This will add the hydrus dev\'s two repositories to your client.', self, self._AutoRepoSetup )
ClientGUIMenus.AppendMenuItem( self, dont_know, 'just set up some repositories for me, please', 'This will add the hydrus dev\'s two repositories to your client.', self._AutoRepoSetup )
ClientGUIMenus.AppendMenu( menu, dont_know, 'I don\'t know what I am doing' )
links = wx.Menu()
site = ClientGUIMenus.AppendMenuBitmapItem( links, 'site', 'Open hydrus\'s website, which is mostly a mirror of the local help.', self, CC.GlobalBMPs.file_repository, webbrowser.open, 'https://hydrusnetwork.github.io/hydrus/' )
site = ClientGUIMenus.AppendMenuBitmapItem( links, '8chan board', 'Open hydrus dev\'s 8chan board, where he makes release posts and other status updates. Much other discussion also occurs.', self, CC.GlobalBMPs.eight_chan, webbrowser.open, 'https://8ch.net/hydrus/index.html' )
site = ClientGUIMenus.AppendMenuBitmapItem( links, 'twitter', 'Open hydrus dev\'s twitter, where he makes general progress updates and emergency notifications.', self, CC.GlobalBMPs.twitter, webbrowser.open, 'https://twitter.com/hydrusnetwork' )
site = ClientGUIMenus.AppendMenuBitmapItem( links, 'tumblr', 'Open hydrus dev\'s tumblr, where he makes release posts and other status updates.', self, CC.GlobalBMPs.tumblr, webbrowser.open, 'http://hydrus.tumblr.com/' )
site = ClientGUIMenus.AppendMenuBitmapItem( links, 'discord', 'Open a discord channel where many hydrus users congregate. Hydrus dev visits regularly.', self, CC.GlobalBMPs.discord, webbrowser.open, 'https://discord.gg/vy8CUB4' )
site = ClientGUIMenus.AppendMenuBitmapItem( links, 'patreon', 'Open hydrus dev\'s patreon, which lets you support development.', self, CC.GlobalBMPs.patreon, webbrowser.open, 'https://www.patreon.com/hydrus_dev' )
site = ClientGUIMenus.AppendMenuBitmapItem( self, links, 'site', 'Open hydrus\'s website, which is mostly a mirror of the local help.', CC.GlobalBMPs.file_repository, webbrowser.open, 'https://hydrusnetwork.github.io/hydrus/' )
site = ClientGUIMenus.AppendMenuBitmapItem( self, links, '8chan board', 'Open hydrus dev\'s 8chan board, where he makes release posts and other status updates. Much other discussion also occurs.', CC.GlobalBMPs.eight_chan, webbrowser.open, 'https://8ch.net/hydrus/index.html' )
site = ClientGUIMenus.AppendMenuBitmapItem( self, links, 'twitter', 'Open hydrus dev\'s twitter, where he makes general progress updates and emergency notifications.', CC.GlobalBMPs.twitter, webbrowser.open, 'https://twitter.com/hydrusnetwork' )
site = ClientGUIMenus.AppendMenuBitmapItem( self, links, 'tumblr', 'Open hydrus dev\'s tumblr, where he makes release posts and other status updates.', CC.GlobalBMPs.tumblr, webbrowser.open, 'http://hydrus.tumblr.com/' )
site = ClientGUIMenus.AppendMenuBitmapItem( self, links, 'discord', 'Open a discord channel where many hydrus users congregate. Hydrus dev visits regularly.', CC.GlobalBMPs.discord, webbrowser.open, 'https://discord.gg/vy8CUB4' )
site = ClientGUIMenus.AppendMenuBitmapItem( self, links, 'patreon', 'Open hydrus dev\'s patreon, which lets you support development.', CC.GlobalBMPs.patreon, webbrowser.open, 'https://www.patreon.com/hydrus_dev' )
ClientGUIMenus.AppendMenu( menu, links, 'links' )
debug = wx.Menu()
ClientGUIMenus.AppendMenuItem( debug, 'make some popups', 'Throw some varied popups at the message manager, just to check it is working.', self, self._DebugMakeSomePopups )
ClientGUIMenus.AppendMenuItem( debug, 'make a popup in five seconds', 'Throw a delayed popup at the message manager, giving you time to minimise or otherwise alter the client before it arrives.', self, wx.CallLater, 5000, HydrusData.ShowText, 'This is a delayed popup message.' )
ClientGUIMenus.AppendMenuCheckItem( debug, 'db report mode', 'Have the db report query information, where supported.', self, HydrusGlobals.db_report_mode, self._SwitchBoolean, 'db_report_mode' )
ClientGUIMenus.AppendMenuCheckItem( debug, 'db profile mode', 'Run detailed \'profiles\' on every database query and dump this information to the log (this is very useful for hydrus dev to have, if something is running slow for you!).', self, HydrusGlobals.db_profile_mode, self._SwitchBoolean, 'db_profile_mode' )
ClientGUIMenus.AppendMenuCheckItem( debug, 'pubsub profile mode', 'Run detailed \'profiles\' on every internal publisher/subscriber message and dump this information to the log. This can hammer your log with dozens of large dumps every second. Don\'t run it unless you know you need to.', self, HydrusGlobals.pubsub_profile_mode, self._SwitchBoolean, 'pubsub_profile_mode' )
ClientGUIMenus.AppendMenuCheckItem( debug, 'force idle mode', 'Make the client consider itself idle and fire all maintenance routines right now. This may hang the gui for a while.', self, HydrusGlobals.force_idle_mode, self._SwitchBoolean, 'force_idle_mode' )
ClientGUIMenus.AppendMenuItem( debug, 'print garbage', 'Print some information about the python garbage to the log.', self, self._DebugPrintGarbage )
ClientGUIMenus.AppendMenuItem( debug, 'clear image rendering cache', 'Tell the image rendering system to forget all current images. This will often free up a bunch of memory immediately.', self, self._controller.ClearCaches )
ClientGUIMenus.AppendMenuItem( debug, 'clear db service info cache', 'Delete all cached service info like total number of mappings or files, in case it has become desynchronised. Some parts of the gui may be laggy immediately after this as these numbers are recalculated.', self, self._DeleteServiceInfo )
ClientGUIMenus.AppendMenuItem( debug, 'load whole db in disk cache', 'Contiguously read as much of the db as will fit into memory. This will massively speed up any subsequent big job.', self, self._controller.CallToThread, self._controller.Read, 'load_into_disk_cache' )
ClientGUIMenus.AppendMenuItem( debug, 'run and initialise server for testing', 'This will try to boot the server in your install folder and initialise it. This is mostly here for testing purposes.', self, self._AutoServerSetup )
ClientGUIMenus.AppendMenuItem( self, debug, 'make some popups', 'Throw some varied popups at the message manager, just to check it is working.', self._DebugMakeSomePopups )
ClientGUIMenus.AppendMenuItem( self, debug, 'make a popup in five seconds', 'Throw a delayed popup at the message manager, giving you time to minimise or otherwise alter the client before it arrives.', wx.CallLater, 5000, HydrusData.ShowText, 'This is a delayed popup message.' )
ClientGUIMenus.AppendMenuCheckItem( self, debug, 'db report mode', 'Have the db report query information, where supported.', HydrusGlobals.db_report_mode, self._SwitchBoolean, 'db_report_mode' )
ClientGUIMenus.AppendMenuCheckItem( self, debug, 'db profile mode', 'Run detailed \'profiles\' on every database query and dump this information to the log (this is very useful for hydrus dev to have, if something is running slow for you!).', HydrusGlobals.db_profile_mode, self._SwitchBoolean, 'db_profile_mode' )
ClientGUIMenus.AppendMenuCheckItem( self, debug, 'pubsub profile mode', 'Run detailed \'profiles\' on every internal publisher/subscriber message and dump this information to the log. This can hammer your log with dozens of large dumps every second. Don\'t run it unless you know you need to.', HydrusGlobals.pubsub_profile_mode, self._SwitchBoolean, 'pubsub_profile_mode' )
ClientGUIMenus.AppendMenuCheckItem( self, debug, 'force idle mode', 'Make the client consider itself idle and fire all maintenance routines right now. This may hang the gui for a while.', HydrusGlobals.force_idle_mode, self._SwitchBoolean, 'force_idle_mode' )
ClientGUIMenus.AppendMenuItem( self, debug, 'print garbage', 'Print some information about the python garbage to the log.', self._DebugPrintGarbage )
ClientGUIMenus.AppendMenuItem( self, debug, 'clear image rendering cache', 'Tell the image rendering system to forget all current images. This will often free up a bunch of memory immediately.', self._controller.ClearCaches )
ClientGUIMenus.AppendMenuItem( self, debug, 'clear db service info cache', 'Delete all cached service info like total number of mappings or files, in case it has become desynchronised. Some parts of the gui may be laggy immediately after this as these numbers are recalculated.', self._DeleteServiceInfo )
ClientGUIMenus.AppendMenuItem( self, debug, 'load whole db in disk cache', 'Contiguously read as much of the db as will fit into memory. This will massively speed up any subsequent big job.', self._controller.CallToThread, self._controller.Read, 'load_into_disk_cache' )
ClientGUIMenus.AppendMenuItem( self, debug, 'run and initialise server for testing', 'This will try to boot the server in your install folder and initialise it. This is mostly here for testing purposes.', self._AutoServerSetup )
ClientGUIMenus.AppendMenu( menu, debug, 'debug' )
ClientGUIMenus.AppendMenuItem( menu, 'hardcoded shortcuts', 'Review some currently hardcoded shortcuts.', self, wx.MessageBox, CC.SHORTCUT_HELP )
ClientGUIMenus.AppendMenuItem( menu, 'about', 'See this client\'s version and other information.', self, self._AboutWindow )
ClientGUIMenus.AppendMenuItem( self, menu, 'hardcoded shortcuts', 'Review some currently hardcoded shortcuts.', wx.MessageBox, CC.SHORTCUT_HELP )
ClientGUIMenus.AppendMenuItem( self, menu, 'about', 'See this client\'s version and other information.', self._AboutWindow )
return ( menu, p( '&Help' ), True )
@ -1527,25 +1528,6 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
self._controller.CallToThread( do_it )
def _MaintainSimilarFilesData( self ):
text = 'This will rebalance the similar files search data, improving search speed.'
text += os.linesep * 2
text += 'If there is work to do, it will report its status through a popup message. The gui may hang until it is done.'
with ClientGUIDialogs.DialogYesNo( self, text ) as dlg:
result = dlg.ShowModal()
if result == wx.ID_YES:
stop_time = HydrusData.GetNow() + 60 * 10
self._controller.Write( 'maintain_similar_files_tree', stop_time = stop_time )
def _ManageAccountTypes( self, service_key ):
with ClientGUIDialogsManage.DialogManageAccountTypes( self, service_key ) as dlg: dlg.ShowModal()
@ -2259,7 +2241,17 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
def _UnclosePage( self, closed_page_index ):
def _UnclosePage( self, closed_page_index = None ):
if closed_page_index is None:
if len( self._closed_pages ) == 0:
return
closed_page_index = 0
with self._lock:
@ -2712,6 +2704,10 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
elif command == 'synchronised_wait_switch': self._SetSynchronisedWait()
elif command == 'tab_menu_close_page': self._ClosePage( self._tab_right_click_index )
elif command == 'tab_menu_rename_page': self._RenamePage( self._tab_right_click_index )
elif command == 'unclose_page':
self._UnclosePage()
elif command == 'undo': self._controller.pub( 'undo' )
else: event.Skip()
@ -2993,7 +2989,7 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
def RefreshAcceleratorTable( self ):
interested_actions = [ 'archive', 'inbox', 'close_page', 'filter', 'manage_ratings', 'manage_tags', 'new_page', 'refresh', 'set_media_focus', 'set_search_focus', 'show_hide_splitters', 'synchronised_wait_switch', 'undo', 'redo' ]
interested_actions = [ 'archive', 'inbox', 'close_page', 'filter', 'manage_ratings', 'manage_tags', 'new_page', 'unclose_page', 'refresh', 'set_media_focus', 'set_search_focus', 'show_hide_splitters', 'synchronised_wait_switch', 'undo', 'redo' ]
entries = []

123
include/ClientGUICommon.py
View File

@ -187,11 +187,11 @@ class AnimatedStaticTextTimestamp( wx.StaticText ):
class BetterBitmapButton( wx.BitmapButton ):
def __init__( self, parent, bitmap, callable, *args, **kwargs ):
def __init__( self, parent, bitmap, func, *args, **kwargs ):
wx.BitmapButton.__init__( self, parent, bitmap = bitmap )
self._callable = callable
self._func = func
self._args = args
self._kwargs = kwargs
self.Bind( wx.EVT_BUTTON, self.EventButton )
@ -199,16 +199,16 @@ class BetterBitmapButton( wx.BitmapButton ):
def EventButton( self, event ):
self._callable( *self._args, **self._kwargs )
self._func( *self._args, **self._kwargs )
class BetterButton( wx.Button ):
def __init__( self, parent, label, callable, *args, **kwargs ):
def __init__( self, parent, label, func, *args, **kwargs ):
wx.Button.__init__( self, parent, label = label )
self._callable = callable
self._func = func
self._args = args
self._kwargs = kwargs
self.Bind( wx.EVT_BUTTON, self.EventButton )
@ -216,7 +216,7 @@ class BetterButton( wx.Button ):
def EventButton( self, event ):
self._callable( *self._args, **self._kwargs )
self._func( *self._args, **self._kwargs )
class BetterChoice( wx.Choice ):
@ -761,7 +761,11 @@ class Gauge( wx.Gauge ):
def SetRange( self, max ):
if max > 1000:
if max is None:
self.Pulse()
elif max > 1000:
self._actual_max = max
wx.Gauge.SetRange( self, 1000 )
@ -775,8 +779,18 @@ class Gauge( wx.Gauge ):
def SetValue( self, value ):
if self._actual_max is None: wx.Gauge.SetValue( self, value )
else: wx.Gauge.SetValue( self, min( int( 1000 * ( float( value ) / self._actual_max ) ), 1000 ) )
if value is None:
self.Pulse()
elif self._actual_max is None:
wx.Gauge.SetValue( self, value )
else:
wx.Gauge.SetValue( self, min( int( 1000 * ( float( value ) / self._actual_max ) ), 1000 ) )
class ListBook( wx.Panel ):
@ -3233,13 +3247,39 @@ class MenuBitmapButton( BetterBitmapButton ):
self._menu_items = menu_items
def _DoBooloanCheck( self, boolean_name ):
new_options = HydrusGlobals.client_controller.GetNewOptions()
new_options.InvertBoolean( boolean_name )
def DoMenu( self ):
menu = wx.Menu()
for ( title, description, callable ) in self._menu_items:
for ( item_type, title, description, data ) in self._menu_items:
ClientGUIMenus.AppendMenuItem( menu, title, description, self, callable )
if item_type == 'normal':
callable = data
ClientGUIMenus.AppendMenuItem( self, menu, title, description, callable )
elif item_type == 'check':
new_options = HydrusGlobals.client_controller.GetNewOptions()
boolean_name = data
initial_value = new_options.GetBoolean( boolean_name )
ClientGUIMenus.AppendMenuCheckItem( self, menu, title, description, initial_value, self._DoBooloanCheck, boolean_name )
elif item_type == 'separator':
menu.AppendSeparator()
HydrusGlobals.client_controller.PopupMenu( self, menu )
@ -3254,13 +3294,39 @@ class MenuButton( BetterButton ):
self._menu_items = menu_items
def _DoBooloanCheck( self, boolean_name ):
new_options = HydrusGlobals.client_controller.GetNewOptions()
new_options.InvertBoolean( boolean_name )
def DoMenu( self ):
menu = wx.Menu()
for ( title, description, callable ) in self._menu_items:
for ( item_type, title, description, data ) in self._menu_items:
ClientGUIMenus.AppendMenuItem( menu, title, description, self, callable )
if item_type == 'normal':
callable = data
ClientGUIMenus.AppendMenuItem( self, menu, title, description, callable )
elif item_type == 'check':
new_options = HydrusGlobals.client_controller.GetNewOptions()
boolean_name = data
initial_value = new_options.GetBoolean( boolean_name )
ClientGUIMenus.AppendMenuCheckItem( self, menu, title, description, initial_value, self._DoBooloanCheck, boolean_name )
elif item_type == 'separator':
menu.AppendSeparator()
HydrusGlobals.client_controller.PopupMenu( self, menu )
@ -3520,6 +3586,10 @@ class PopupMessage( PopupWindow ):
self._gauge_2.Bind( wx.EVT_RIGHT_DOWN, self.EventDismiss )
self._gauge_2.Hide()
self._download = TextAndGauge( self )
self._download.Bind( wx.EVT_RIGHT_DOWN, self.EventDismiss )
self._download.Hide()
self._copy_to_clipboard_button = wx.Button( self )
self._copy_to_clipboard_button.Bind( wx.EVT_BUTTON, self.EventCopyToClipboardButton )
self._copy_to_clipboard_button.Bind( wx.EVT_RIGHT_DOWN, self.EventDismiss )
@ -3565,6 +3635,7 @@ class PopupMessage( PopupWindow ):
vbox.AddF( self._gauge_1, CC.FLAGS_EXPAND_PERPENDICULAR )
vbox.AddF( self._text_2, CC.FLAGS_EXPAND_PERPENDICULAR )
vbox.AddF( self._gauge_2, CC.FLAGS_EXPAND_PERPENDICULAR )
vbox.AddF( self._download, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
vbox.AddF( self._copy_to_clipboard_button, CC.FLAGS_EXPAND_PERPENDICULAR )
vbox.AddF( self._show_files_button, CC.FLAGS_EXPAND_PERPENDICULAR )
vbox.AddF( self._show_tb_button, CC.FLAGS_EXPAND_PERPENDICULAR )
@ -3792,6 +3863,21 @@ class PopupMessage( PopupWindow ):
self._gauge_2.Hide()
popup_download = self._job_key.GetIfHasVariable( 'popup_download' )
if popup_download is not None:
( text, gauge_value, gauge_range ) = popup_download
self._download.SetValue( text, gauge_value, gauge_range )
self._download.Show()
else:
self._download.Hide()
popup_clipboard = self._job_key.GetIfHasVariable( 'popup_clipboard' )
if popup_clipboard is not None:
@ -5627,8 +5713,15 @@ class TextAndGauge( wx.Panel ):
self._st.SetLabelText( text )
self._gauge.SetRange( range )
self._gauge.SetValue( value )
if value is None or range is None:
self._gauge.Pulse()
else:
self._gauge.SetRange( range )
self._gauge.SetValue( value )
( TimeDeltaEvent, EVT_TIME_DELTA ) = wx.lib.newevent.NewCommandEvent()

50
include/ClientGUIDialogs.py
View File

@ -2242,7 +2242,7 @@ class DialogInputShortcut( Dialog ):
self._shortcut = ClientGUICommon.Shortcut( self, modifier, key )
self._actions = wx.Choice( self, choices = [ 'archive', 'inbox', 'close_page', 'filter', 'fullscreen_switch', 'frame_back', 'frame_next', 'manage_ratings', 'manage_tags', 'new_page', 'refresh', 'set_media_focus', 'set_search_focus', 'show_hide_splitters', 'synchronised_wait_switch', 'next', 'first', 'last', 'undo', 'redo', 'open_externally', 'pan_up', 'pan_down', 'pan_left', 'pan_right', 'previous', 'remove', 'zoom_in', 'zoom_out', 'zoom_switch' ] )
self._actions = wx.Choice( self, choices = [ 'archive', 'inbox', 'close_page', 'filter', 'fullscreen_switch', 'frame_back', 'frame_next', 'manage_ratings', 'manage_tags', 'new_page', 'unclose_page', 'refresh', 'set_media_focus', 'set_search_focus', 'show_hide_splitters', 'synchronised_wait_switch', 'next', 'first', 'last', 'undo', 'redo', 'open_externally', 'pan_up', 'pan_down', 'pan_left', 'pan_right', 'previous', 'remove', 'zoom_in', 'zoom_out', 'zoom_switch' ] )
self._ok = wx.Button( self, id= wx.ID_OK, label = 'Ok' )
self._ok.SetForegroundColour( ( 0, 128, 0 ) )
@ -4340,6 +4340,8 @@ class DialogSetupExport( Dialog ):
self._tags_box = ClientGUICommon.StaticBoxSorterForListBoxTags( self, 'files\' tags' )
self._tag_txt_tag_services = []
t = ClientGUICommon.ListBoxTagsSelection( self._tags_box, include_counts = True, collapse_siblings = True )
self._tags_box.SetTagsBox( t )
@ -4371,6 +4373,7 @@ class DialogSetupExport( Dialog ):
self._export_tag_txts = wx.CheckBox( self, label = 'export tags in .txt files?' )
self._export_tag_txts.SetToolTipString( text )
self._export_tag_txts.Bind( wx.EVT_CHECKBOX, self.EventExportTagTxtsChanged )
self._export = wx.Button( self, label = 'export' )
self._export.Bind( wx.EVT_BUTTON, self.EventExport )
@ -4525,11 +4528,18 @@ class DialogSetupExport( Dialog ):
tags_manager = media.GetTagsManager()
tags = tags_manager.GetCurrent()
tags = set()
filename = ClientExporting.GenerateExportFilename( media, terms )
for service_key in self._tag_txt_tag_services:
tags.update( tags_manager.GetCurrent( service_key ) )
txt_path = os.path.join( directory, filename + '.txt' )
tags = list( tags )
tags.sort()
txt_path = path + '.txt'
with open( txt_path, 'wb' ) as f:
@ -4564,6 +4574,38 @@ class DialogSetupExport( Dialog ):
HydrusGlobals.client_controller.CallToThread( do_it )
def EventExportTagTxtsChanged( self, event ):
if self._export_tag_txts.GetValue() == True:
services_manager = HydrusGlobals.client_controller.GetServicesManager()
tag_services = services_manager.GetServices( HC.TAG_SERVICES )
names_to_service_keys = { service.GetName() : service.GetServiceKey() for service in tag_services }
service_keys_to_names = { service_key : name for ( name, service_key ) in names_to_service_keys.items() }
tag_service_names = names_to_service_keys.keys()
tag_service_names.sort()
with DialogCheckFromListOfStrings( self, 'select tag services', tag_service_names, self._tag_txt_tag_services ) as dlg:
if dlg.ShowModal() == wx.ID_OK:
selected_names = dlg.GetChecked()
self._tag_txt_tag_services = [ names_to_service_keys[ name ] for name in selected_names ]
else:
self._export_tag_txts.SetValue( False )
def EventOpenLocation( self, event ):
directory = self._directory_picker.GetPath()

2
include/ClientGUIDialogsManage.py
View File

@ -3181,6 +3181,8 @@ class DialogManagePixivAccount( ClientGUIDialogs.Dialog ):
except HydrusExceptions.ForbiddenException as e:
HydrusData.ShowException( e )
self._status.SetLabelText( 'Did not work! ' + repr( e ) )

62
include/ClientGUIManagement.py
View File

@ -1450,8 +1450,12 @@ class ManagementPanelDuplicateFilter( ManagementPanel ):
menu_items = []
menu_items.append( ( 'refresh', 'This panel does not update itself when files are added or deleted elsewhere in the client. Hitting this will refresh the numbers from the database.', self._RefreshAndUpdateStatus ) )
menu_items.append( ( 'reset potentials', 'This will delete all the potential duplicate pairs found so far and reset their files\' search status.', self._ResetUnknown ) )
menu_items.append( ( 'normal', 'refresh', 'This panel does not update itself when files are added or deleted elsewhere in the client. Hitting this will refresh the numbers from the database.', self._RefreshAndUpdateStatus ) )
menu_items.append( ( 'normal', 'reset potential duplicates', 'This will delete all the potential duplicate pairs found so far and reset their files\' search status.', self._ResetUnknown ) )
menu_items.append( ( 'separator', 0, 0, 0 ) )
menu_items.append( ( 'check', 'regenerate file information in normal db maintenance', 'Tell the client to include file phash regeneration in its normal db maintenance cycles, whether you have that set to idle or shutdown time.', 'maintain_similar_files_phashes_during_idle' ) )
menu_items.append( ( 'check', 'rebalance tree in normal db maintenance', 'Tell the client to balance the tree in its normal db maintenance cycles, whether you have that set to idle or shutdown time. It will not occur whille there are phashes still to regenerate.', 'maintain_similar_files_tree_during_idle' ) )
menu_items.append( ( 'check', 'find duplicate pairs at the current distance in normal db maintenance', 'Tell the client to find duplicate pairs in its normal db maintenance cycles, whether you have that set to idle or shutdown time. It will not occur whille there are phashes still to regenerate or if the tree still needs rebalancing.', 'maintain_similar_files_duplicate_pairs_during_idle' ) )
self._cog_button = ClientGUICommon.MenuBitmapButton( self, CC.GlobalBMPs.cog, menu_items )
@ -1471,10 +1475,10 @@ class ManagementPanelDuplicateFilter( ManagementPanel ):
menu_items = []
menu_items.append( ( 'exact match', 'Search for exact matches.', self._SetSearchDistanceExact ) )
menu_items.append( ( 'very similar', 'Search for very similar files.', self._SetSearchDistanceVerySimilar ) )
menu_items.append( ( 'similar', 'Search for similar files.', self._SetSearchDistanceSimilar ) )
menu_items.append( ( 'speculative', 'Search for files that are probably similar.', self._SetSearchDistanceSpeculative ) )
menu_items.append( ( 'normal', 'exact match', 'Search for exact matches.', HydrusData.Call( self._SetSearchDistance, HC.HAMMING_EXACT_MATCH ) ) )
menu_items.append( ( 'normal', 'very similar', 'Search for very similar files.', HydrusData.Call( self._SetSearchDistance, HC.HAMMING_VERY_SIMILAR ) ) )
menu_items.append( ( 'normal', 'similar', 'Search for similar files.', HydrusData.Call( self._SetSearchDistance, HC.HAMMING_SIMILAR ) ) )
menu_items.append( ( 'normal', 'speculative', 'Search for files that are probably similar.', HydrusData.Call( self._SetSearchDistance, HC.HAMMING_SPECULATIVE ) ) )
self._search_distance_button = ClientGUICommon.MenuButton( self._searching_panel, 'similarity', menu_items )
@ -1492,6 +1496,7 @@ class ManagementPanelDuplicateFilter( ManagementPanel ):
self._num_unknown_duplicates = wx.StaticText( self._filtering_panel )
self._num_same_file_duplicates = wx.StaticText( self._filtering_panel )
self._num_alternate_duplicates = wx.StaticText( self._filtering_panel )
self._show_some_dupes = ClientGUICommon.BetterButton( self._filtering_panel, 'show some pairs (prototype!)', self._ShowSomeDupes )
#
@ -1526,7 +1531,7 @@ class ManagementPanelDuplicateFilter( ManagementPanel ):
gridbox_2.AddGrowableCol( 0, 1 )
gridbox_2.AddF( self._num_searched, CC.FLAGS_EXPAND_PERPENDICULAR )
gridbox_2.AddF( self._num_searched, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
gridbox_2.AddF( self._search_button, CC.FLAGS_VCENTER )
self._searching_panel.AddF( distance_hbox, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
@ -1537,6 +1542,7 @@ class ManagementPanelDuplicateFilter( ManagementPanel ):
self._filtering_panel.AddF( self._num_unknown_duplicates, CC.FLAGS_EXPAND_PERPENDICULAR )
self._filtering_panel.AddF( self._num_same_file_duplicates, CC.FLAGS_EXPAND_PERPENDICULAR )
self._filtering_panel.AddF( self._num_alternate_duplicates, CC.FLAGS_EXPAND_PERPENDICULAR )
self._filtering_panel.AddF( self._show_some_dupes, CC.FLAGS_EXPAND_PERPENDICULAR )
#
@ -1601,24 +1607,15 @@ class ManagementPanelDuplicateFilter( ManagementPanel ):
self._UpdateStatus()
def _SetSearchDistanceExact( self ):
def _ShowSomeDupes( self ):
self._SetSearchDistance( HC.HAMMING_EXACT_MATCH )
hashes = self._controller.Read( 'some_dupes' )
def _SetSearchDistanceSimilar( self ):
media_results = self._controller.Read( 'media_results', hashes )
self._SetSearchDistance( HC.HAMMING_SIMILAR )
panel = ClientGUIMedia.MediaPanelThumbnails( self._page, self._page_key, CC.COMBINED_LOCAL_FILE_SERVICE_KEY, media_results )
def _SetSearchDistanceSpeculative( self ):
self._SetSearchDistance( HC.HAMMING_SPECULATIVE )
def _SetSearchDistanceVerySimilar( self ):
self._SetSearchDistance( HC.HAMMING_VERY_SIMILAR )
self._controller.pub( 'swap_media_panel', self._page_key, panel )
def _StartStopDBJob( self ):
@ -1631,6 +1628,7 @@ class ManagementPanelDuplicateFilter( ManagementPanel ):
self._search_button.Disable()
self._search_distance_button.Disable()
self._search_distance_spinctrl.Disable()
self._show_some_dupes.Disable()
self._job_key = ClientThreading.JobKey( cancellable = True )
@ -1668,6 +1666,17 @@ class ManagementPanelDuplicateFilter( ManagementPanel ):
def _UpdateJob( self ):
if self._job_key.TimeRunning() > 30:
self._job_key.Cancel()
self._job_key = None
self._StartStopDBJob()
return
if self._job_key.IsDone():
self._job_key = None
@ -1740,7 +1749,7 @@ class ManagementPanelDuplicateFilter( ManagementPanel ):
num_done = total_num_files - num_phashes_to_regen
self._num_phashes_to_regen.SetLabelText( HydrusData.ConvertValueRangeToPrettyString( num_done, total_num_files ) + 'eligible files up to date.' )
self._num_phashes_to_regen.SetLabelText( HydrusData.ConvertValueRangeToPrettyString( num_done, total_num_files ) + ' eligible files up to date.' )
self._phashes_button.Enable()
@ -1806,6 +1815,15 @@ class ManagementPanelDuplicateFilter( ManagementPanel ):
self._num_same_file_duplicates.SetLabelText( HydrusData.ConvertIntToPrettyString( duplicate_types_to_count[ HC.DUPLICATE_SAME_FILE ] ) + ' same file pairs filtered.' )
self._num_alternate_duplicates.SetLabelText( HydrusData.ConvertIntToPrettyString( duplicate_types_to_count[ HC.DUPLICATE_ALTERNATE ] ) + ' alternate file pairs filtered.' )
if num_unknown > 0:
self._show_some_dupes.Enable()
else:
self._show_some_dupes.Disable()
def EventSearchDistanceChanged( self, event ):

96
include/ClientGUIMedia.py
View File

@ -61,7 +61,7 @@ def AddServiceKeyLabelsToMenu( menu, service_keys, phrase ):
ClientGUIMenus.AppendMenu( menu, submenu, phrase + u'\u2026' )
def AddServiceKeysToMenu( menu, service_keys, phrase, description, event_handler, callable ):
def AddServiceKeysToMenu( event_handler, menu, service_keys, phrase, description, callable ):
services_manager = HydrusGlobals.client_controller.GetServicesManager()
@ -73,7 +73,7 @@ def AddServiceKeysToMenu( menu, service_keys, phrase, description, event_handler
label = phrase + ' ' + name
ClientGUIMenus.AppendMenuItem( menu, label, description, event_handler, callable, service_key )
ClientGUIMenus.AppendMenuItem( event_handler, menu, label, description, callable, service_key )
else:
@ -83,7 +83,7 @@ def AddServiceKeysToMenu( menu, service_keys, phrase, description, event_handler
name = services_manager.GetName( service_key )
ClientGUIMenus.AppendMenuItem( submenu, name, description, event_handler, callable, service_key )
ClientGUIMenus.AppendMenuItem( event_handler, submenu, name, description, callable, service_key )
ClientGUIMenus.AppendMenu( menu, submenu, phrase + u'\u2026' )
@ -2334,7 +2334,7 @@ class MediaPanelThumbnails( MediaPanel ):
if thumbnail is None:
ClientGUIMenus.AppendMenuItem( menu, 'refresh', 'Refresh the current search.', self, HydrusGlobals.client_controller.pub, 'refresh_query', self._page_key )
ClientGUIMenus.AppendMenuItem( self, menu, 'refresh', 'Refresh the current search.', HydrusGlobals.client_controller.pub, 'refresh_query', self._page_key )
if len( self._sorted_media ) > 0:
@ -2344,15 +2344,15 @@ class MediaPanelThumbnails( MediaPanel ):
if len( self._selected_media ) < len( self._sorted_media ):
ClientGUIMenus.AppendMenuItem( select_menu, 'all', 'Select everything.', self, self._Select, 'all' )
ClientGUIMenus.AppendMenuItem( self, select_menu, 'all', 'Select everything.', self._Select, 'all' )
ClientGUIMenus.AppendMenuItem( select_menu, 'invert', 'Swap what is and is not selected.', self, self._Select, 'invert' )
ClientGUIMenus.AppendMenuItem( self, select_menu, 'invert', 'Swap what is and is not selected.', self._Select, 'invert' )
if media_has_archive and media_has_inbox:
ClientGUIMenus.AppendMenuItem( select_menu, 'inbox', 'Select everything in the inbox.', self, self._Select, 'inbox' )
ClientGUIMenus.AppendMenuItem( select_menu, 'archive', 'Select everything that is archived.', self, self._Select, 'archive' )
ClientGUIMenus.AppendMenuItem( self, select_menu, 'inbox', 'Select everything in the inbox.', self._Select, 'inbox' )
ClientGUIMenus.AppendMenuItem( self, select_menu, 'archive', 'Select everything that is archived.', self._Select, 'archive' )
if len( all_specific_file_domains ) > 1:
@ -2370,13 +2370,13 @@ class MediaPanelThumbnails( MediaPanel ):
name = services_manager.GetName( service_key )
ClientGUIMenus.AppendMenuItem( select_menu, name, 'Select everything in ' + name + '.', self, self._Select, 'file_service', service_key )
ClientGUIMenus.AppendMenuItem( self, select_menu, name, 'Select everything in ' + name + '.', self._Select, 'file_service', service_key )
if len( self._selected_media ) > 0:
ClientGUIMenus.AppendMenuItem( select_menu, 'none', 'Deselect everything.', self, self._Select, 'none' )
ClientGUIMenus.AppendMenuItem( self, select_menu, 'none', 'Deselect everything.', self._Select, 'none' )
ClientGUIMenus.AppendMenu( menu, select_menu, 'select' )
@ -2705,67 +2705,67 @@ class MediaPanelThumbnails( MediaPanel ):
if len( downloadable_file_service_keys ) > 0:
ClientGUIMenus.AppendMenuItem( remote_action_menu, download_phrase, 'Download all possible selected files.', self, self._DownloadSelected )
ClientGUIMenus.AppendMenuItem( self, remote_action_menu, download_phrase, 'Download all possible selected files.', self._DownloadSelected )
if some_downloading:
ClientGUIMenus.AppendMenuItem( remote_action_menu, rescind_download_phrase, 'Stop downloading any of the selected files.', self, self._RescindDownloadSelected )
ClientGUIMenus.AppendMenuItem( self, remote_action_menu, rescind_download_phrase, 'Stop downloading any of the selected files.', self._RescindDownloadSelected )
if len( uploadable_file_service_keys ) > 0:
AddServiceKeysToMenu( remote_action_menu, uploadable_file_service_keys, upload_phrase, 'Upload all selected files to the file repository.', self, self._UploadFiles )
AddServiceKeysToMenu( self, remote_action_menu, uploadable_file_service_keys, upload_phrase, 'Upload all selected files to the file repository.', self._UploadFiles )
if len( pending_file_service_keys ) > 0:
AddServiceKeysToMenu( remote_action_menu, pending_file_service_keys, rescind_upload_phrase, 'Rescind the pending upload to the file repository.', self, self._RescindUploadFiles )
AddServiceKeysToMenu( self, remote_action_menu, pending_file_service_keys, rescind_upload_phrase, 'Rescind the pending upload to the file repository.', self._RescindUploadFiles )
if len( petitionable_file_service_keys ) > 0:
AddServiceKeysToMenu( remote_action_menu, petitionable_file_service_keys, petition_phrase, 'Petition these files for deletion from the file repository.', self, self._PetitionFiles )
AddServiceKeysToMenu( self, remote_action_menu, petitionable_file_service_keys, petition_phrase, 'Petition these files for deletion from the file repository.', self._PetitionFiles )
if len( petitioned_file_service_keys ) > 0:
AddServiceKeysToMenu( remote_action_menu, petitioned_file_service_keys, rescind_petition_phrase, 'Rescind the petition to delete these files from the file repository.', self, self._RescindPetitionFiles )
AddServiceKeysToMenu( self, remote_action_menu, petitioned_file_service_keys, rescind_petition_phrase, 'Rescind the petition to delete these files from the file repository.', self._RescindPetitionFiles )
if len( deletable_file_service_keys ) > 0:
AddServiceKeysToMenu( remote_action_menu, deletable_file_service_keys, remote_delete_phrase, 'Delete these files from the file repository.', self, self._Delete )
AddServiceKeysToMenu( self, remote_action_menu, deletable_file_service_keys, remote_delete_phrase, 'Delete these files from the file repository.', self._Delete )
if len( modifyable_file_service_keys ) > 0:
AddServiceKeysToMenu( remote_action_menu, modifyable_file_service_keys, modify_account_phrase, 'Modify the account(s) that uploaded these files to the file repository.', self, self._ModifyUploaders )
AddServiceKeysToMenu( self, remote_action_menu, modifyable_file_service_keys, modify_account_phrase, 'Modify the account(s) that uploaded these files to the file repository.', self._ModifyUploaders )
if len( pinnable_ipfs_service_keys ) > 0:
AddServiceKeysToMenu( remote_action_menu, pinnable_ipfs_service_keys, pin_phrase, 'Pin these files to the ipfs service.', self, self._UploadFiles )
AddServiceKeysToMenu( self, remote_action_menu, pinnable_ipfs_service_keys, pin_phrase, 'Pin these files to the ipfs service.', self._UploadFiles )
if len( pending_ipfs_service_keys ) > 0:
AddServiceKeysToMenu( remote_action_menu, pending_ipfs_service_keys, rescind_pin_phrase, 'Rescind the pending pin to the ipfs service.', self, self._RescindUploadFiles )
AddServiceKeysToMenu( self, remote_action_menu, pending_ipfs_service_keys, rescind_pin_phrase, 'Rescind the pending pin to the ipfs service.', self._RescindUploadFiles )
if len( unpinnable_ipfs_service_keys ) > 0:
AddServiceKeysToMenu( remote_action_menu, unpinnable_ipfs_service_keys, unpin_phrase, 'Unpin these files from the ipfs service.', self, self._PetitionFiles )
AddServiceKeysToMenu( self, remote_action_menu, unpinnable_ipfs_service_keys, unpin_phrase, 'Unpin these files from the ipfs service.', self._PetitionFiles )
if len( petitioned_ipfs_service_keys ) > 0:
AddServiceKeysToMenu( remote_action_menu, petitioned_ipfs_service_keys, rescind_unpin_phrase, 'Rescind the pending unpin from the ipfs service.', self, self._RescindPetitionFiles )
AddServiceKeysToMenu( self, remote_action_menu, petitioned_ipfs_service_keys, rescind_unpin_phrase, 'Rescind the pending unpin from the ipfs service.', self._RescindPetitionFiles )
if multiple_selected and len( ipfs_service_keys ) > 0:
AddServiceKeysToMenu( remote_action_menu, ipfs_service_keys, 'pin new directory to', 'Pin these files as a directory to the ipfs service.', self, self._UploadDirectory )
AddServiceKeysToMenu( self, remote_action_menu, ipfs_service_keys, 'pin new directory to', 'Pin these files as a directory to the ipfs service.', self._UploadDirectory )
ClientGUIMenus.AppendMenu( menu, remote_action_menu, 'remote services' )
@ -2777,8 +2777,8 @@ class MediaPanelThumbnails( MediaPanel ):
manage_menu = wx.Menu()
ClientGUIMenus.AppendMenuItem( manage_menu, manage_tags_phrase, 'Manage tags for the selected files.', self, self._ManageTags )
ClientGUIMenus.AppendMenuItem( manage_menu, manage_ratings_phrase, 'Manage ratings for the selected files.', self, self._ManageRatings )
ClientGUIMenus.AppendMenuItem( self, manage_menu, manage_tags_phrase, 'Manage tags for the selected files.', self._ManageTags )
ClientGUIMenus.AppendMenuItem( self, manage_menu, manage_ratings_phrase, 'Manage ratings for the selected files.', self._ManageRatings )
ClientGUIMenus.AppendMenu( menu, manage_menu, 'manage' )
@ -2793,7 +2793,7 @@ class MediaPanelThumbnails( MediaPanel ):
phrase = 'manage file\'s tags'
ClientGUIMenus.AppendMenuItem( menu, phrase, 'Manage tags for the selected files.', self, self._ManageTags )
ClientGUIMenus.AppendMenuItem( self, menu, phrase, 'Manage tags for the selected files.', self._ManageTags )
#
@ -2804,7 +2804,7 @@ class MediaPanelThumbnails( MediaPanel ):
if selection_has_local_file_domain:
ClientGUIMenus.AppendMenuItem( filter_menu, 'archive/delete', 'Launch a special media viewer that will quickly archive (left-click) and delete (right-click) the selected media.', self, self._Filter )
ClientGUIMenus.AppendMenuItem( self, filter_menu, 'archive/delete', 'Launch a special media viewer that will quickly archive (left-click) and delete (right-click) the selected media.', self._Filter )
shortcut_names = HydrusGlobals.client_controller.Read( 'serialisable_names', HydrusSerialisable.SERIALISABLE_TYPE_SHORTCUTS )
@ -2813,20 +2813,20 @@ class MediaPanelThumbnails( MediaPanel ):
custom_shortcuts_menu = wx.Menu()
ClientGUIMenus.AppendMenuItem( custom_shortcuts_menu, 'manage', 'Manage your different custom filters and their shortcuts.', self, self._CustomFilter )
ClientGUIMenus.AppendMenuItem( self, custom_shortcuts_menu, 'manage', 'Manage your different custom filters and their shortcuts.', self._CustomFilter )
custom_shortcuts_menu.AppendSeparator()
for shortcut_name in shortcut_names:
ClientGUIMenus.AppendMenuItem( custom_shortcuts_menu, shortcut_name, 'Open the ' + shortcut_name + ' custom filter.', self, self._CustomFilter, shortcut_name )
ClientGUIMenus.AppendMenuItem( self, custom_shortcuts_menu, shortcut_name, 'Open the ' + shortcut_name + ' custom filter.', self._CustomFilter, shortcut_name )
ClientGUIMenus.AppendMenu( filter_menu, custom_shortcuts_menu, 'custom filters' )
else:
ClientGUIMenus.AppendMenuItem( filter_menu, 'create a custom filter', 'Create a custom filter that uses non-default shortcuts.', self, self._CustomFilter )
ClientGUIMenus.AppendMenuItem( self, filter_menu, 'create a custom filter', 'Create a custom filter that uses non-default shortcuts.', self._CustomFilter )
ClientGUIMenus.AppendMenu( menu, filter_menu, 'filter' )
@ -2836,25 +2836,25 @@ class MediaPanelThumbnails( MediaPanel ):
if selection_has_inbox:
ClientGUIMenus.AppendMenuItem( menu, archive_phrase, 'Archive the selected files.', self, self._Archive )
ClientGUIMenus.AppendMenuItem( self, menu, archive_phrase, 'Archive the selected files.', self._Archive )
if selection_has_archive:
ClientGUIMenus.AppendMenuItem( menu, inbox_phrase, 'Put the selected files back in the inbox.', self, self._Inbox )
ClientGUIMenus.AppendMenuItem( self, menu, inbox_phrase, 'Put the selected files back in the inbox.', self._Inbox )
ClientGUIMenus.AppendMenuItem( menu, remove_phrase, 'Remove the selected files from the current view.', self, self._Remove )
ClientGUIMenus.AppendMenuItem( self, menu, remove_phrase, 'Remove the selected files from the current view.', self._Remove )
if selection_has_local_file_domain:
ClientGUIMenus.AppendMenuItem( menu, local_delete_phrase, 'Delete the selected files from \'my files\'.', self, self._Delete, CC.LOCAL_FILE_SERVICE_KEY )
ClientGUIMenus.AppendMenuItem( self, menu, local_delete_phrase, 'Delete the selected files from \'my files\'.', self._Delete, CC.LOCAL_FILE_SERVICE_KEY )
if selection_has_trash:
ClientGUIMenus.AppendMenuItem( menu, trash_delete_phrase, 'Delete the selected files from the trash, forcing an immediate physical delete from your hard drive.', self, self._Delete, CC.TRASH_SERVICE_KEY )
ClientGUIMenus.AppendMenuItem( menu, undelete_phrase, 'Restore the selected files back to \'my files\'.', self, self._Undelete )
ClientGUIMenus.AppendMenuItem( self, menu, trash_delete_phrase, 'Delete the selected files from the trash, forcing an immediate physical delete from your hard drive.', self._Delete, CC.TRASH_SERVICE_KEY )
ClientGUIMenus.AppendMenuItem( self, menu, undelete_phrase, 'Restore the selected files back to \'my files\'.', self._Undelete )
# share
@ -2863,7 +2863,7 @@ class MediaPanelThumbnails( MediaPanel ):
if selection_has_local:
ClientGUIMenus.AppendMenuItem( menu, 'open externally', 'Launch this file with your OS\'s default program for it.', self, self._OpenExternally )
ClientGUIMenus.AppendMenuItem( self, menu, 'open externally', 'Launch this file with your OS\'s default program for it.', self._OpenExternally )
share_menu = wx.Menu()
@ -2975,7 +2975,7 @@ class MediaPanelThumbnails( MediaPanel ):
menu.AppendSeparator()
ClientGUIMenus.AppendMenuItem( menu, 'refresh', 'Refresh the current search.', self, HydrusGlobals.client_controller.pub, 'refresh_query', self._page_key )
ClientGUIMenus.AppendMenuItem( self, menu, 'refresh', 'Refresh the current search.', HydrusGlobals.client_controller.pub, 'refresh_query', self._page_key )
if len( self._sorted_media ) > 0:
@ -2985,15 +2985,15 @@ class MediaPanelThumbnails( MediaPanel ):
if len( self._selected_media ) < len( self._sorted_media ):
ClientGUIMenus.AppendMenuItem( select_menu, 'all', 'Select everything.', self, self._Select, 'all' )
ClientGUIMenus.AppendMenuItem( self, select_menu, 'all', 'Select everything.', self._Select, 'all' )
ClientGUIMenus.AppendMenuItem( select_menu, 'invert', 'Swap what is and is not selected.', self, self._Select, 'invert' )
ClientGUIMenus.AppendMenuItem( self, select_menu, 'invert', 'Swap what is and is not selected.', self._Select, 'invert' )
if media_has_archive and media_has_inbox:
ClientGUIMenus.AppendMenuItem( select_menu, 'inbox', 'Select everything in the inbox.', self, self._Select, 'inbox' )
ClientGUIMenus.AppendMenuItem( select_menu, 'archive', 'Select everything that is archived.', self, self._Select, 'archive' )
ClientGUIMenus.AppendMenuItem( self, select_menu, 'inbox', 'Select everything in the inbox.', self._Select, 'inbox' )
ClientGUIMenus.AppendMenuItem( self, select_menu, 'archive', 'Select everything that is archived.', self._Select, 'archive' )
if len( all_specific_file_domains ) > 1:
@ -3011,13 +3011,13 @@ class MediaPanelThumbnails( MediaPanel ):
name = services_manager.GetName( service_key )
ClientGUIMenus.AppendMenuItem( select_menu, name, 'Select everything in ' + name + '.', self, self._Select, 'file_service', service_key )
ClientGUIMenus.AppendMenuItem( self, select_menu, name, 'Select everything in ' + name + '.', self._Select, 'file_service', service_key )
if len( self._selected_media ) > 0:
ClientGUIMenus.AppendMenuItem( select_menu, 'none', 'Deselect everything.', self, self._Select, 'none' )
ClientGUIMenus.AppendMenuItem( self, select_menu, 'none', 'Deselect everything.', self._Select, 'none' )
ClientGUIMenus.AppendMenu( menu, select_menu, 'select' )
@ -3033,10 +3033,10 @@ class MediaPanelThumbnails( MediaPanel ):
similar_menu = wx.Menu()
ClientGUIMenus.AppendMenuItem( similar_menu, 'exact match', 'Search the database for files that look precisely like this one.', self, self._GetSimilarTo, HC.HAMMING_EXACT_MATCH )
ClientGUIMenus.AppendMenuItem( similar_menu, 'very similar', 'Search the database for files that look just like this one.', self, self._GetSimilarTo, HC.HAMMING_VERY_SIMILAR )
ClientGUIMenus.AppendMenuItem( similar_menu, 'similar', 'Search the database for files that look generally like this one.', self, self._GetSimilarTo, HC.HAMMING_SIMILAR )
ClientGUIMenus.AppendMenuItem( similar_menu, 'speculative', 'Search the database for files that probably look like this one. This is sometimes useful for symbols with sharp edges or lines.', self, self._GetSimilarTo, HC.HAMMING_SPECULATIVE )
ClientGUIMenus.AppendMenuItem( self, similar_menu, 'exact match', 'Search the database for files that look precisely like this one.', self._GetSimilarTo, HC.HAMMING_EXACT_MATCH )
ClientGUIMenus.AppendMenuItem( self, similar_menu, 'very similar', 'Search the database for files that look just like this one.', self._GetSimilarTo, HC.HAMMING_VERY_SIMILAR )
ClientGUIMenus.AppendMenuItem( self, similar_menu, 'similar', 'Search the database for files that look generally like this one.', self._GetSimilarTo, HC.HAMMING_SIMILAR )
ClientGUIMenus.AppendMenuItem( self, similar_menu, 'speculative', 'Search the database for files that probably look like this one. This is sometimes useful for symbols with sharp edges or lines.', self._GetSimilarTo, HC.HAMMING_SPECULATIVE )
ClientGUIMenus.AppendMenu( menu, similar_menu, 'find similar files' )

17
include/ClientGUIMenus.py
View File

@ -1,5 +1,6 @@
import ClientConstants as CC
import collections
import HydrusGlobals
import wx
menus_to_submenus = collections.defaultdict( set )
@ -13,7 +14,7 @@ def AppendMenu( menu, submenu, label ):
menus_to_submenus[ menu ].add( submenu )
def AppendMenuBitmapItem( menu, label, description, event_handler, bitmap, callable, *args, **kwargs ):
def AppendMenuBitmapItem( event_handler, menu, label, description, bitmap, callable, *args, **kwargs ):
label = SanitiseLabel( label )
@ -25,11 +26,11 @@ def AppendMenuBitmapItem( menu, label, description, event_handler, bitmap, calla
menu.AppendItem( menu_item )
BindMenuItem( menu, event_handler, menu_item, callable, *args, **kwargs )
BindMenuItem( event_handler, menu, menu_item, callable, *args, **kwargs )
return menu_item
def AppendMenuCheckItem( menu, label, description, event_handler, initial_value, callable, *args, **kwargs ):
def AppendMenuCheckItem( event_handler, menu, label, description, initial_value, callable, *args, **kwargs ):
label = SanitiseLabel( label )
@ -37,17 +38,17 @@ def AppendMenuCheckItem( menu, label, description, event_handler, initial_value,
menu_item.Check( initial_value )
BindMenuItem( menu, event_handler, menu_item, callable, *args, **kwargs )
BindMenuItem( event_handler, menu, menu_item, callable, *args, **kwargs )
return menu_item
def AppendMenuItem( menu, label, description, event_handler, callable, *args, **kwargs ):
def AppendMenuItem( event_handler, menu, label, description, callable, *args, **kwargs ):
label = SanitiseLabel( label )
menu_item = menu.Append( wx.ID_ANY, label, description )
BindMenuItem( menu, event_handler, menu_item, callable, *args, **kwargs )
BindMenuItem( event_handler, menu, menu_item, callable, *args, **kwargs )
return menu_item
@ -62,7 +63,7 @@ def AppendMenuLabel( menu, label, description = None ):
return menu_item
def BindMenuItem( menu, event_handler, menu_item, callable, *args, **kwargs ):
def BindMenuItem( event_handler, menu, menu_item, callable, *args, **kwargs ):
l_callable = GetLambdaCallable( callable, *args, **kwargs )
@ -72,6 +73,8 @@ def BindMenuItem( menu, event_handler, menu_item, callable, *args, **kwargs ):
def DestroyMenuItems( menu ):
handler = HydrusGlobals.client_controller.GetApp()
menu_item_data = menus_to_menu_item_data[ menu ]
del menus_to_menu_item_data[ menu ]

16
include/ClientGUIParsing.py
View File

@ -406,8 +406,8 @@ class EditNodes( wx.Panel ):
menu_items = []
menu_items.append( ( 'content node', 'A node that parses the given data for content.', self.AddContentNode ) )
menu_items.append( ( 'link node', 'A node that parses the given data for a link, which it then pursues.', self.AddLinkNode ) )
menu_items.append( ( 'normal', 'content node', 'A node that parses the given data for content.', self.AddContentNode ) )
menu_items.append( ( 'normal', 'link node', 'A node that parses the given data for a link, which it then pursues.', self.AddLinkNode ) )
self._add_button = ClientGUICommon.MenuButton( self, 'add', menu_items )
@ -1559,21 +1559,21 @@ class ManageParsingScriptsPanel( ClientGUIScrolledPanels.ManagePanel ):
menu_items = []
menu_items.append( ( 'file lookup script', 'A script that fetches content for a known file.', self.AddFileLookupScript ) )
menu_items.append( ( 'normal', 'file lookup script', 'A script that fetches content for a known file.', self.AddFileLookupScript ) )
self._add_button = ClientGUICommon.MenuButton( self, 'add', menu_items )
menu_items = []
menu_items.append( ( 'to clipboard', 'Serialise the script and put it on your clipboard.', self.ExportToClipboard ) )
menu_items.append( ( 'to png', 'Serialise the script and encode it to an image file you can easily share with other hydrus users.', self.ExportToPng ) )
menu_items.append( ( 'normal', 'to clipboard', 'Serialise the script and put it on your clipboard.', self.ExportToClipboard ) )
menu_items.append( ( 'normal', 'to png', 'Serialise the script and encode it to an image file you can easily share with other hydrus users.', self.ExportToPng ) )
self._export_button = ClientGUICommon.MenuButton( self, 'export', menu_items )
menu_items = []
menu_items.append( ( 'from clipboard', 'Load a script from text in your clipboard.', self.ImportFromClipboard ) )
menu_items.append( ( 'from png', 'Load a script from an encoded png.', self.ImportFromPng ) )
menu_items.append( ( 'normal', 'from clipboard', 'Load a script from text in your clipboard.', self.ImportFromClipboard ) )
menu_items.append( ( 'normal', 'from png', 'Load a script from an encoded png.', self.ImportFromPng ) )
self._import_button = ClientGUICommon.MenuButton( self, 'import', menu_items )
@ -2052,7 +2052,7 @@ class ScriptManagementControl( wx.Panel ):
for url in urls:
ClientGUIMenus.AppendMenuItem( menu, url, 'launch this url in your browser', self, webbrowser.open, url )
ClientGUIMenus.AppendMenuItem( self, menu, url, 'launch this url in your browser', webbrowser.open, url )
HydrusGlobals.client_controller.PopupMenu( self, menu )

11
include/ClientGUIScrolledPanelsManagement.py
View File

@ -2372,15 +2372,15 @@ class ManageSubscriptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
menu_items = []
menu_items.append( ( 'to clipboard', 'Serialise the script and put it on your clipboard.', self.ExportToClipboard ) )
menu_items.append( ( 'to png', 'Serialise the script and encode it to an image file you can easily share with other hydrus users.', self.ExportToPng ) )
menu_items.append( ( 'normal', 'to clipboard', 'Serialise the script and put it on your clipboard.', self.ExportToClipboard ) )
menu_items.append( ( 'normal', 'to png', 'Serialise the script and encode it to an image file you can easily share with other hydrus users.', self.ExportToPng ) )
self._export = ClientGUICommon.MenuButton( self, 'export', menu_items )
menu_items = []
menu_items.append( ( 'from clipboard', 'Load a script from text in your clipboard.', self.ImportFromClipboard ) )
menu_items.append( ( 'from png', 'Load a script from an encoded png.', self.ImportFromPng ) )
menu_items.append( ( 'normal', 'from clipboard', 'Load a script from text in your clipboard.', self.ImportFromClipboard ) )
menu_items.append( ( 'normal', 'from png', 'Load a script from an encoded png.', self.ImportFromPng ) )
self._import = ClientGUICommon.MenuButton( self, 'import', menu_items )
@ -2439,7 +2439,8 @@ class ManageSubscriptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
( name, gallery_identifier, gallery_stream_identifiers, query, period, get_tags_if_redundant, initial_file_limit, periodic_file_limit, paused, import_file_options, import_tag_options, last_checked, last_error, check_now, seed_cache ) = subscription.ToTuple()
pretty_site = HC.site_type_string_lookup[ gallery_identifier.GetSiteType() ]
pretty_site = gallery_identifier.ToString()
pretty_last_checked = HydrusData.ConvertTimestampToPrettySync( last_checked )
pretty_period = HydrusData.ConvertTimeDeltaToPrettyString( period )

130
include/ClientGUITopLevelWindows.py
View File

@ -274,9 +274,74 @@ class NewDialog( wx.Dialog ):
self.Bind( wx.EVT_BUTTON, self.EventDialogButton )
self.Bind( wx.EVT_MENU_CLOSE, self.EventMenuClose )
self.Bind( wx.EVT_MENU_HIGHLIGHT_ALL, self.EventMenuHighlight )
self.Bind( wx.EVT_MENU_OPEN, self.EventMenuOpen )
self._menu_stack = []
self._menu_text_stack = []
HydrusGlobals.client_controller.ResetIdleTimer()
def EventMenuClose( self, event ):
menu = event.GetMenu()
if menu is not None and menu in self._menu_stack:
index = self._menu_stack.index( menu )
del self._menu_stack[ index ]
previous_text = self._menu_text_stack.pop()
status_bar = HydrusGlobals.client_controller.GetGUI().GetStatusBar()
status_bar.SetStatusText( previous_text )
def EventMenuHighlight( self, event ):
status_bar = HydrusGlobals.client_controller.GetGUI().GetStatusBar()
if len( self._menu_stack ) > 0:
text = ''
menu = self._menu_stack[-1]
if menu is not None:
menu_item = menu.FindItemById( event.GetMenuId() )
if menu_item is not None:
text = menu_item.GetHelp()
status_bar.SetStatusText( text )
def EventMenuOpen( self, event ):
menu = event.GetMenu()
if menu is not None:
status_bar = HydrusGlobals.client_controller.GetGUI().GetStatusBar()
previous_text = status_bar.GetStatusText()
self._menu_stack.append( menu )
self._menu_text_stack.append( previous_text )
def EventDialogButton( self, event ): self.EndModal( event.GetId() )
class DialogThatResizes( NewDialog ):
@ -474,9 +539,74 @@ class Frame( wx.Frame ):
self.SetIcon( wx.Icon( os.path.join( HC.STATIC_DIR, 'hydrus.ico' ), wx.BITMAP_TYPE_ICO ) )
self.Bind( wx.EVT_MENU_CLOSE, self.EventMenuClose )
self.Bind( wx.EVT_MENU_HIGHLIGHT_ALL, self.EventMenuHighlight )
self.Bind( wx.EVT_MENU_OPEN, self.EventMenuOpen )
self._menu_stack = []
self._menu_text_stack = []
HydrusGlobals.client_controller.ResetIdleTimer()
def EventMenuClose( self, event ):
menu = event.GetMenu()
if menu is not None and menu in self._menu_stack:
index = self._menu_stack.index( menu )
del self._menu_stack[ index ]
previous_text = self._menu_text_stack.pop()
status_bar = HydrusGlobals.client_controller.GetGUI().GetStatusBar()
status_bar.SetStatusText( previous_text )
def EventMenuHighlight( self, event ):
status_bar = HydrusGlobals.client_controller.GetGUI().GetStatusBar()
if len( self._menu_stack ) > 0:
text = ''
menu = self._menu_stack[-1]
if menu is not None:
menu_item = menu.FindItemById( event.GetMenuId() )
if menu_item is not None:
text = menu_item.GetHelp()
status_bar.SetStatusText( text )
def EventMenuOpen( self, event ):
menu = event.GetMenu()
if menu is not None:
status_bar = HydrusGlobals.client_controller.GetGUI().GetStatusBar()
previous_text = status_bar.GetStatusText()
self._menu_stack.append( menu )
self._menu_text_stack.append( previous_text )
class FrameThatResizes( Frame ):
def __init__( self, parent, title, frame_key, float_on_parent = True ):

17
include/ClientImporting.py
View File

@ -372,7 +372,7 @@ class GalleryImport( HydrusSerialisable.SerialisableBase ):
else:
text = str( e )
text = HydrusData.ToUnicode( e )
HydrusData.DebugPrint( traceback.format_exc() )
@ -726,6 +726,21 @@ class HDDImport( HydrusSerialisable.SerialisableBase ):
HydrusData.ShowException( e )
txt_path = path + '.txt'
if os.path.exists( txt_path ):
try:
ClientData.DeletePath( txt_path )
except Exception as e:
HydrusData.ShowText( 'While attempting to delete ' + txt_path + ', the following error occured:' )
HydrusData.ShowException( e )
except HydrusExceptions.MimeException as e:

157
include/ClientMedia.py
View File

@ -424,6 +424,8 @@ class MediaList( object ):
def _GetSortFunction( self, sort_by ):
reverse = False
( sort_by_type, sort_by_data ) = sort_by
def deal_with_none( x ):
@ -434,11 +436,28 @@ class MediaList( object ):
if sort_by_type == 'system':
if sort_by_data == CC.SORT_BY_RANDOM: sort_function = lambda x: random.random()
elif sort_by_data == CC.SORT_BY_SMALLEST: sort_function = lambda x: deal_with_none( x.GetSize() )
elif sort_by_data == CC.SORT_BY_LARGEST: sort_function = lambda x: -deal_with_none( x.GetSize() )
elif sort_by_data == CC.SORT_BY_SHORTEST: sort_function = lambda x: deal_with_none( x.GetDuration() )
elif sort_by_data == CC.SORT_BY_LONGEST: sort_function = lambda x: -deal_with_none( x.GetDuration() )
if sort_by_data == CC.SORT_BY_RANDOM:
sort_function = lambda x: random.random()
elif sort_by_data in ( CC.SORT_BY_SMALLEST, CC.SORT_BY_LARGEST ):
sort_function = lambda x: deal_with_none( x.GetSize() )
if sort_by_data == CC.SORT_BY_LARGEST:
reverse = True
elif sort_by_data in ( CC.SORT_BY_SHORTEST, CC.SORT_BY_LONGEST ):
sort_function = lambda x: deal_with_none( x.GetDuration() )
if sort_by_data == CC.SORT_BY_LONGEST:
reverse = True
elif sort_by_data in ( CC.SORT_BY_OLDEST, CC.SORT_BY_NEWEST ):
file_service = HydrusGlobals.client_controller.GetServicesManager().GetService( self._file_service_key )
@ -454,10 +473,77 @@ class MediaList( object ):
file_service_key = self._file_service_key
if sort_by_data == CC.SORT_BY_OLDEST: sort_function = lambda x: deal_with_none( x.GetTimestamp( file_service_key ) )
elif sort_by_data == CC.SORT_BY_NEWEST: sort_function = lambda x: -deal_with_none( x.GetTimestamp( file_service_key ) )
sort_function = lambda x: deal_with_none( x.GetTimestamp( file_service_key ) )
if sort_by_data == CC.SORT_BY_NEWEST:
reverse = True
elif sort_by_data in ( CC.SORT_BY_HEIGHT_ASC, CC.SORT_BY_HEIGHT_DESC ):
sort_function = lambda x: deal_with_none( x.GetResolution()[0] )
if sort_by_data == CC.SORT_BY_HEIGHT_DESC:
reverse = True
elif sort_by_data in ( CC.SORT_BY_WIDTH_ASC, CC.SORT_BY_WIDTH_DESC ):
sort_function = lambda x: deal_with_none( x.GetResolution()[1] )
if sort_by_data == CC.SORT_BY_WIDTH_DESC:
reverse = True
elif sort_by_data in ( CC.SORT_BY_RATIO_ASC, CC.SORT_BY_RATIO_DESC ):
def sort_function( x ):
( width, height ) = x.GetResolution()
if width is None or height is None or width == 0 or height == 0:
return -1
else:
return float( width ) / float( height )
if sort_by_data == CC.SORT_BY_RATIO_DESC:
reverse = True
elif sort_by_data in ( CC.SORT_BY_NUM_PIXELS_ASC, CC.SORT_BY_NUM_PIXELS_DESC ):
def sort_function( x ):
( width, height ) = x.GetResolution()
if width is None or height is None:
return -1
else:
return width * height
if sort_by_data == CC.SORT_BY_NUM_PIXELS_DESC:
reverse = True
elif sort_by_data == CC.SORT_BY_MIME:
sort_function = lambda x: x.GetMime()
elif sort_by_data == CC.SORT_BY_MIME: sort_function = lambda x: x.GetMime()
elif sort_by_type == 'namespaces':
@ -474,26 +560,24 @@ class MediaList( object ):
service_key = sort_by_data
def ratings_sort_function( service_key, reverse, x ):
def ratings_sort_function( service_key, x ):
x_ratings_manager = x.GetRatingsManager()
rating = deal_with_none( x_ratings_manager.GetRating( service_key ) )
if reverse:
rating *= -1
return rating
reverse = sort_by_type == 'rating_descend'
sort_function = lambda x: ratings_sort_function( service_key, x )
sort_function = lambda x: ratings_sort_function( service_key, reverse, x )
if sort_by_type == 'rating_descend':
reverse = True
return sort_function
return ( sort_function, reverse )
def _RecalcHashes( self ):
@ -889,15 +973,15 @@ class MediaList( object ):
sort_by_fallback = sort_choices[ 0 ]
sort_function = self._GetSortFunction( sort_by_fallback )
( sort_function, reverse ) = self._GetSortFunction( sort_by_fallback )
self._sorted_media.sort( sort_function )
self._sorted_media.sort( sort_function, reverse = reverse )
# this is a stable sort, so the fallback order above will remain for equal items
sort_function = self._GetSortFunction( self._sort_by )
( sort_function, reverse ) = self._GetSortFunction( self._sort_by )
self._sorted_media.sort( sort_function )
self._sorted_media.sort( sort_function = sort_function, reverse = reverse )
class ListeningMediaList( MediaList ):
@ -1590,28 +1674,17 @@ class MediaResult( object ):
class SortedList( object ):
def __init__( self, initial_items = None, sort_function = None ):
def __init__( self, initial_items = None ):
if initial_items is None: initial_items = []
do_sort = sort_function is not None
if sort_function is None:
sort_function = lambda x: x
self._sort_function = sort_function
self._sort_function = lambda x: x
self._sort_reverse = False
self._sorted_list = list( initial_items )
self._items_to_indices = None
if do_sort:
self.sort()
def __contains__( self, item ):
@ -1700,14 +1773,20 @@ class SortedList( object ):
self._DirtyIndices()
def sort( self, f = None ):
def sort( self, sort_function = None, reverse = False ):
if f is not None:
if sort_function is None:
self._sort_function = f
sort_function = self._sort_function
reverse = self._sort_reverse
else:
self._sort_function = sort_function
self._sort_reverse = reverse
self._sorted_list.sort( key = f )
self._sorted_list.sort( key = sort_function, reverse = reverse )
self._DirtyIndices()

74
include/ClientNetworking.py
View File

@ -6,6 +6,7 @@ import errno
import httplib
import os
import requests
from requests.packages.urllib3.exceptions import InsecureRequestWarning
import socket
import socks
import ssl
@ -18,6 +19,8 @@ import HydrusData
import itertools
import HydrusGlobals
requests.packages.urllib3.disable_warnings( InsecureRequestWarning )
def AddHydrusCredentialsToHeaders( credentials, request_headers ):
if credentials.HasAccessKey():
@ -240,6 +243,51 @@ def SetProxy( proxytype, host, port, username = None, password = None ):
socks.wrapmodule( httplib )
def StreamResponseToFile( job_key, response, f ):
if 'content-length' in response.headers:
gauge_range = int( response.headers[ 'content-length' ] )
else:
gauge_range = None
gauge_value = 0
try:
for chunk in response.iter_content( chunk_size = 65536 ):
( i_paused, should_quit ) = job_key.WaitIfNeeded()
if should_quit:
raise HydrusExceptions.CancelledException()
f.write( chunk )
gauge_value += len( chunk )
if gauge_range is None:
text = 'downloading - ' + HydrusData.ConvertIntToBytes( gauge_value )
else:
text = 'downloading - ' + HydrusData.ConvertValueRangeToBytes( gauge_value, gauge_range )
job_key.SetVariable( 'popup_download', ( text, gauge_value, gauge_range ) )
finally:
job_key.DeleteVariable( 'popup_download' )
class HTTPConnectionManager( object ):
def __init__( self ):
@ -522,6 +570,11 @@ class HTTPConnection( object ):
request_headers[ 'User-Agent' ] = 'hydrus/' + str( HC.NETWORK_VERSION )
if 'Accept' not in request_headers:
request_headers[ 'Accept' ] = '*/*'
path_and_query = HydrusData.ToByteString( path_and_query )
request_headers = { str( k ) : str( v ) for ( k, v ) in request_headers.items() }
@ -568,13 +621,6 @@ class HTTPConnection( object ):
if attempt_number <= 3:
if self._hydrus_network:
# we are talking to a new hydrus server, which uses https, and hence an http call gives badstatusline
self._scheme = 'https'
self._RefreshConnection()
return self._GetInitialResponse( method, path_and_query, request_headers, body, attempt_number = attempt_number + 1 )
@ -616,13 +662,6 @@ class HTTPConnection( object ):
if attempt_number <= 3:
if self._hydrus_network:
# we are talking to a new hydrus server, which uses https, and hence an http call gives badstatusline
self._scheme = 'https'
self._RefreshConnection()
return self._GetInitialResponse( method, path_and_query, request_headers, body, attempt_number = attempt_number + 1 )
@ -645,13 +684,6 @@ class HTTPConnection( object ):
if attempt_number <= 3:
if self._hydrus_network:
# we are talking to a new hydrus server, which uses https, and hence an http call gives badstatusline
self._scheme = 'https'
self._RefreshConnection()
return self._GetInitialResponse( method_string, path_and_query, request_headers, body, attempt_number = attempt_number + 1 )

7
include/ClientThreading.py
View File

@ -21,6 +21,8 @@ class JobKey( object ):
self._only_start_if_unbusy = only_start_if_unbusy
self._stop_time = stop_time
self._start_time = HydrusData.GetNow()
self._deleted = threading.Event()
self._deletion_time = None
self._begun = threading.Event()
@ -273,6 +275,11 @@ class JobKey( object ):
time.sleep( 0.00001 )
def TimeRunning( self ):
return HydrusData.GetNow() - self._start_time
def ToString( self ):
stuff_to_print = []

95
include/ClientVPTree.py
View File

@ -1,95 +0,0 @@
import random
import HydrusData
class VPTreeNode( object ):
def __init__( self, phashes ):
ghd = HydrusData.GetHammingDistance
if len( phashes ) == 1:
( self._phash, ) = phashes
self._radius = 0
inner_phashes = []
outer_phashes = []
else:
# we want to choose a good node.
# a good node is one that doesn't overlap with other circles much
# get a random sample with big lists, to keep cpu costs down
if len( phashes ) > 50: phashes_sample = random.sample( phashes, 50 )
else: phashes_sample = phashes
all_nodes_comparisons = { phash1 : [ ( ghd( phash1, phash2 ), phash2 ) for phash2 in phashes_sample if phash2 != phash1 ] for phash1 in phashes_sample }
for comparisons in all_nodes_comparisons.values(): comparisons.sort()
# the median of the sorted hamming distances makes a decent radius
all_nodes_radii = [ ( comparisons[ len( comparisons ) / 2 ], phash ) for ( phash, comparisons ) in all_nodes_comparisons.items() ]
all_nodes_radii.sort()
# let's make our node the phash with the smallest predicted radius
( ( predicted_radius, whatever ), self._phash ) = all_nodes_radii[ 0 ]
if len( phashes ) > 50:
my_hammings = [ ( ghd( self._phash, phash ), phash ) for phash in phashes if phash != self._phash ]
my_hammings.sort()
else:
my_hammings = all_nodes_comparisons[ self._phash ]
median_index = len( my_hammings ) / 2
( self._radius, whatever ) = my_hammings[ median_index ]
# lets bump our index up until we actually get outside the radius
while median_index + 1 < len( my_hammings ) and my_hammings[ median_index + 1 ][0] == self._radius: median_index += 1
# now separate my phashes into inside and outside that radius
inner_phashes = [ phash for ( hamming, phash ) in my_hammings[ : median_index + 1 ] ]
outer_phashes = [ phash for ( hamming, phash ) in my_hammings[ median_index + 1 : ] ]
if len( inner_phashes ) == 0: self._inner_node = VPTreeNodeEmpty()
else: self._inner_node = VPTreeNode( inner_phashes )
if len( outer_phashes ) == 0: self._outer_node = VPTreeNodeEmpty()
else: self._outer_node = VPTreeNode( outer_phashes )
def __len__( self ): return len( self._inner_node ) + len( self._outer_node ) + 1
def GetMatches( self, phash, max_hamming ):
hamming_distance_to_me = HydrusData.GetHammingDistance( self._phash, phash )
matches = []
if hamming_distance_to_me <= max_hamming: matches.append( self._phash )
if hamming_distance_to_me <= ( self._radius + max_hamming ): matches.extend( self._inner_node.GetMatches( phash, max_hamming ) ) # i.e. result could be in inner
if hamming_distance_to_me >= ( self._radius - max_hamming ): matches.extend( self._outer_node.GetMatches( phash, max_hamming ) ) # i.e. result could be in outer
return matches
class VPTreeNodeEmpty( object ):
def __init__( self ): pass
def __len__( self ): return 0
def GetMatches( self, phash, max_hamming ): return []

2
include/HydrusConstants.py
View File

@ -49,7 +49,7 @@ options = {}
# Misc
NETWORK_VERSION = 17
SOFTWARE_VERSION = 241
SOFTWARE_VERSION = 242
UNSCALED_THUMBNAIL_DIMENSIONS = ( 200, 200 )

42
include/HydrusDB.py
View File

@ -367,8 +367,6 @@ class HydrusDB( object ):
self._connection_timestamp = HydrusData.GetNow()
self._db.create_function( 'hydrus_hamming', 2, HydrusData.GetHammingDistance )
self._c = self._db.cursor()
self._c.execute( 'PRAGMA main.cache_size = -10000;' )
@ -515,6 +513,44 @@ class HydrusDB( object ):
HydrusData.Print( text )
def _SelectFromList( self, select_statement, xs ):
# issue here is that doing a simple blah_id = ? is real quick and cacheable but doing a lot of fetchone()s is slow
# blah_id IN ( 1, 2, 3 ) is fast to execute but not cacheable and doing the str() list splay takes time so there is initial lag
# doing the temporaryintegertable trick works well for gigantic lists you refer to frequently but it is super laggy when you sometimes are only selecting four things
# blah_id IN ( ?, ?, ? ) is fast and cacheable but there's a small limit (1024 is too many) to the number of params sql can handle
# so lets do the latter but break it into 256-strong chunks to get a good medium
# this will take a select statement with %s like so:
# SELECT blah_id, blah FROM blahs WHERE blah_id IN %s;
MAX_CHUNK_SIZE = 256
# do this just so we aren't always reproducing this long string for gigantic lists
# and also so we aren't overmaking it when this gets spammed with a lot of len() == 1 calls
if len( xs ) >= MAX_CHUNK_SIZE:
max_statement = select_statement % ( '(' + ','.join( '?' * MAX_CHUNK_SIZE ) + ')' )
for chunk in HydrusData.SplitListIntoChunks( xs, MAX_CHUNK_SIZE ):
if len( chunk ) == MAX_CHUNK_SIZE:
chunk_statement = max_statement
else:
chunk_statement = select_statement % ( '(' + ','.join( '?' * len( chunk ) ) + ')' )
for row in self._c.execute( chunk_statement, chunk ):
yield row
def _UpdateDB( self, version ):
raise NotImplementedError()
@ -722,4 +758,4 @@ class TemporaryIntegerTable( object ):
return False

48
include/HydrusData.py
View File

@ -13,6 +13,7 @@ import psutil
import random
import shutil
import sqlite3
import struct
import subprocess
import sys
import threading
@ -685,25 +686,28 @@ def GetEmptyDataDict():
return data
def GetHammingDistance( phash1, phash2 ):
def Get64BitHammingDistance( phash1, phash2 ):
distance = 0
# old way of doing this was:
#while xor > 0:
#
# distance += 1
# xor &= xor - 1
#
phash1 = bytearray( phash1 )
phash2 = bytearray( phash2 )
# convert to unsigned long long, then xor
# then through the power of stackexchange magic, we get number of bits in record time
for i in range( len( phash1 ) ):
xor = phash1[i] ^ phash2[i]
while xor > 0:
distance += 1
xor &= xor - 1
n = struct.unpack( '!Q', phash1 )[0] ^ struct.unpack( '!Q', phash2 )[0]
return distance
n = ( n & 0x5555555555555555 ) + ( ( n & 0xAAAAAAAAAAAAAAAA ) >> 1 ) # 10101010, 01010101
n = ( n & 0x3333333333333333 ) + ( ( n & 0xCCCCCCCCCCCCCCCC ) >> 2 ) # 11001100, 00110011
n = ( n & 0x0F0F0F0F0F0F0F0F ) + ( ( n & 0xF0F0F0F0F0F0F0F0 ) >> 4 ) # 11110000, 00001111
n = ( n & 0x00FF00FF00FF00FF ) + ( ( n & 0xFF00FF00FF00FF00 ) >> 8 ) # etc...
n = ( n & 0x0000FFFF0000FFFF ) + ( ( n & 0xFFFF0000FFFF0000 ) >> 16 )
n = ( n & 0x00000000FFFFFFFF ) + ( ( n & 0xFFFFFFFF00000000 ) >> 32 )
return n
def GetNow(): return int( time.time() )
@ -1510,6 +1514,20 @@ class BigJobPauser( object ):
class Call( object ):
def __init__( self, func, *args, **kwargs ):
self._func = func
self._args = args
self._kwargs = kwargs
def __call__( self ):
self._func( *self._args, **self._kwargs )
class ClientToServerContentUpdatePackage( HydrusYAMLBase ):
yaml_tag = u'!ClientToServerContentUpdatePackage'

3
include/HydrusImageHandling.py
View File

@ -2,7 +2,6 @@ import cStringIO
import HydrusConstants as HC
import HydrusExceptions
import HydrusThreading
import lz4
import os
from PIL import _imaging
from PIL import Image as PILImage
@ -236,4 +235,4 @@ def GetThumbnailResolution( ( im_x, im_y ), ( target_x, target_y ) ):
target_y = int( target_y )
return ( target_x, target_y )

18
include/HydrusNetworking.py
View File

@ -1,7 +1,8 @@
import httplib
import socket
import ssl
def GetLocalConnection( port ):
def GetLocalConnection( port, https = False ):
old_socket = httplib.socket.socket
@ -9,7 +10,18 @@ def GetLocalConnection( port ):
try:
connection = httplib.HTTPConnection( '127.0.0.1', port, timeout = 8 )
if https:
context = ssl.SSLContext( ssl.PROTOCOL_SSLv23 )
context.options |= ssl.OP_NO_SSLv2
context.options |= ssl.OP_NO_SSLv3
connection = httplib.HTTPSConnection( '127.0.0.1', port, timeout = 8, context = context )
else:
connection = httplib.HTTPConnection( '127.0.0.1', port, timeout = 8 )
connection.connect()
@ -19,4 +31,4 @@ def GetLocalConnection( port ):
return connection

2
include/HydrusPaths.py
View File

@ -346,7 +346,7 @@ def MakeFileWritable( path ):
except:
except Exception as e:
pass

12
include/HydrusSerialisable.py
View File

@ -1,5 +1,6 @@
import json
import lz4
import zlib
SERIALISABLE_TYPE_BASE = 0
SERIALISABLE_TYPE_BASE_NAMED = 1
@ -39,7 +40,14 @@ SERIALISABLE_TYPES_TO_OBJECT_TYPES = {}
def CreateFromNetworkString( network_string ):
obj_string = lz4.loads( network_string )
try:
obj_string = zlib.decompress( network_string )
except zlib.error:
obj_string = lz4.loads( network_string )
return CreateFromString( obj_string )
@ -260,4 +268,4 @@ class SerialisableList( SerialisableBase, list ):
SERIALISABLE_TYPES_TO_OBJECT_TYPES[ SERIALISABLE_TYPE_LIST ] = SerialisableList
SERIALISABLE_TYPES_TO_OBJECT_TYPES[ SERIALISABLE_TYPE_LIST ] = SerialisableList

7
include/ServerController.py
View File

@ -89,7 +89,7 @@ def ShutdownSiblingInstance( db_dir ):
try:
connection = HydrusNetworking.GetLocalConnection( port )
connection = HydrusNetworking.GetLocalConnection( port, https = True )
connection.request( 'GET', '/' )
@ -310,7 +310,10 @@ class Controller( HydrusController.HydrusController ):
already_bound = True
except: pass
except:
pass
if already_bound:

6
include/ServerDB.py
View File

@ -11,7 +11,6 @@ import HydrusPaths
import HydrusSerialisable
import itertools
import json
import lz4
import os
import Queue
import random
@ -608,7 +607,10 @@ class DB( HydrusDB.HydrusDB ):
max_monthly_data = options[ 'max_monthly_data' ]
if max_monthly_data is not None and total_used_bytes > max_monthly_data: self._services_over_monthly_data.add( service_key )
if max_monthly_data is not None and total_used_bytes > max_monthly_data:
self._services_over_monthly_data.add( service_key )

2
include/ServerServerResources.py
View File

@ -547,4 +547,4 @@ class HydrusResourceCommandRestrictedServiceUpdate( HydrusResourceCommandRestric
return response_context

10
include/TestClientDownloading.py
View File

@ -181,6 +181,8 @@ class TestDownloaders( unittest.TestCase ):
self.assertEqual( info, expected_info )
'''
# hid this when sankaku started cloudflare 503ing and removed sankaku from the booru defaults
def test_sankaku( self ):
with open( os.path.join( HC.STATIC_DIR, 'testing', 'sankaku_gallery.html' ) ) as f: sankaku_gallery = f.read()
@ -249,7 +251,7 @@ class TestDownloaders( unittest.TestCase ):
self.assertEqual( data, 'swf file' )
'''
def test_booru_e621( self ):
with open( os.path.join( HC.STATIC_DIR, 'testing', 'e621_gallery.html' ) ) as f: e621_gallery = f.read()
@ -360,7 +362,7 @@ class TestDownloaders( unittest.TestCase ):
info = ( data, tags )
expected_info = ('picture', [u'creator:Sparrow', u'title:Ashantae!', u'Shantae', u'Asha', u'Monster_World', u'cosplay', u'nips'])
expected_info = ('picture', [u'creator:Sparrow', u'title:Ashantae!'])
self.assertEqual( info, expected_info )
@ -379,8 +381,8 @@ class TestDownloaders( unittest.TestCase ):
info = ( data, tags )
expected_info = ('scrap', [u'creator:Sparrow', u'title:Swegabe Sketches \u2013 Gabrielle 027', u'bukkake', u'horsecock', u'gokkun', u'prom_night'])
expected_info = ('scrap', [u'creator:Sparrow', u'title:Swegabe Sketches \u2013 Gabrielle 027'])
self.assertEqual( info, expected_info )

4
include/TestDB.py
View File

@ -1196,10 +1196,6 @@ class TestServerDB( unittest.TestCase ):
time.sleep( 0.1 )
# so they can be deleted later on no prob
os.chmod( self._db._ssl_cert_path, stat.S_IREAD | stat.S_IWRITE )
os.chmod( self._db._ssl_key_path, stat.S_IREAD | stat.S_IWRITE )
del self._db

51
include/TestHydrusServer.py
View File

@ -6,6 +6,7 @@ import ClientMedia
import hashlib
import httplib
import HydrusConstants as HC
import HydrusEncryption
import HydrusPaths
import HydrusServer
import HydrusServerResources
@ -15,6 +16,7 @@ import os
import ServerFiles
import ServerServer
import shutil
import ssl
import stat
import TestConstants
import time
@ -23,6 +25,7 @@ import unittest
from twisted.internet import reactor
from twisted.internet.endpoints import TCP4ClientEndpoint, connectProtocol
from twisted.internet.defer import deferredGenerator, waitForDeferred
import twisted.internet.ssl
import HydrusData
import HydrusGlobals
@ -70,11 +73,23 @@ class TestServer( unittest.TestCase ):
def TWISTEDSetup():
reactor.listenTCP( HC.DEFAULT_SERVER_ADMIN_PORT, ServerServer.HydrusServiceAdmin( self._admin_service.GetServiceKey(), HC.SERVER_ADMIN, 'hello' ) )
self._ssl_cert_path = os.path.join( TestConstants.DB_DIR, 'server.crt' )
self._ssl_key_path = os.path.join( TestConstants.DB_DIR, 'server.key' )
# if db test ran, this is still hanging around and read-only, so don't bother to fail overwriting
if not os.path.exists( self._ssl_cert_path ):
HydrusEncryption.GenerateOpenSSLCertAndKeyFile( self._ssl_cert_path, self._ssl_key_path )
context_factory = twisted.internet.ssl.DefaultOpenSSLContextFactory( self._ssl_key_path, self._ssl_cert_path )
reactor.listenSSL( HC.DEFAULT_SERVER_ADMIN_PORT, ServerServer.HydrusServiceAdmin( self._admin_service.GetServiceKey(), HC.SERVER_ADMIN, 'hello' ), context_factory )
reactor.listenSSL( HC.DEFAULT_SERVICE_PORT, ServerServer.HydrusServiceRepositoryFile( self._file_service.GetServiceKey(), HC.FILE_REPOSITORY, 'hello' ), context_factory )
reactor.listenSSL( HC.DEFAULT_SERVICE_PORT + 1, ServerServer.HydrusServiceRepositoryTag( self._tag_service.GetServiceKey(), HC.TAG_REPOSITORY, 'hello' ), context_factory )
reactor.listenTCP( HC.DEFAULT_LOCAL_FILE_PORT, ClientLocalServer.HydrusServiceLocal( CC.COMBINED_LOCAL_FILE_SERVICE_KEY, HC.COMBINED_LOCAL_FILE, 'hello' ) )
reactor.listenTCP( HC.DEFAULT_LOCAL_BOORU_PORT, ClientLocalServer.HydrusServiceBooru( CC.LOCAL_BOORU_SERVICE_KEY, HC.LOCAL_BOORU, 'hello' ) )
reactor.listenTCP( HC.DEFAULT_SERVICE_PORT, ServerServer.HydrusServiceRepositoryFile( self._file_service.GetServiceKey(), HC.FILE_REPOSITORY, 'hello' ) )
reactor.listenTCP( HC.DEFAULT_SERVICE_PORT + 1, ServerServer.HydrusServiceRepositoryTag( self._tag_service.GetServiceKey(), HC.TAG_REPOSITORY, 'hello' ) )
reactor.callFromThread( TWISTEDSetup )
@ -82,16 +97,20 @@ class TestServer( unittest.TestCase ):
time.sleep( 1 )
@classmethod
def tearDownClass( self ):
def _test_basics( self, host, port, https = True ):
shutil.rmtree( ServerFiles.GetExpectedUpdateDir( self._file_service.GetServiceKey() ) )
shutil.rmtree( ServerFiles.GetExpectedUpdateDir( self._tag_service.GetServiceKey() ) )
def _test_basics( self, host, port ):
connection = httplib.HTTPConnection( host, port, timeout = 10 )
if https:
context = ssl.SSLContext( ssl.PROTOCOL_SSLv23 )
context.options |= ssl.OP_NO_SSLv2
context.options |= ssl.OP_NO_SSLv3
connection = httplib.HTTPSConnection( host, port, timeout = 10, context = context )
else:
connection = httplib.HTTPConnection( host, port, timeout = 10 )
#
@ -174,6 +193,8 @@ class TestServer( unittest.TestCase ):
path = ServerFiles.GetExpectedFilePath( self._file_hash )
HydrusPaths.MakeSureDirectoryExists( os.path.dirname( path ) )
with open( path, 'wb' ) as f: f.write( EXAMPLE_FILE )
response = service.Request( HC.GET, 'file', { 'hash' : self._file_hash.encode( 'hex' ) } )
@ -217,6 +238,8 @@ class TestServer( unittest.TestCase ):
path = ServerFiles.GetExpectedThumbnailPath( self._file_hash )
HydrusPaths.MakeSureDirectoryExists( os.path.dirname( path ) )
with open( path, 'wb' ) as f: f.write( EXAMPLE_THUMBNAIL )
response = service.Request( HC.GET, 'thumbnail', { 'hash' : self._file_hash.encode( 'hex' ) } )
@ -593,7 +616,7 @@ class TestServer( unittest.TestCase ):
host = '127.0.0.1'
port = HC.DEFAULT_LOCAL_FILE_PORT
self._test_basics( host, port )
self._test_basics( host, port, https = False )
self._test_local_file( host, port )
@ -649,7 +672,7 @@ class TestServer( unittest.TestCase ):
host = '127.0.0.1'
port = HC.DEFAULT_LOCAL_BOORU_PORT
self._test_basics( host, port )
self._test_basics( host, port, https = False )
self._test_local_booru( host, port )
'''

6
test.py
View File

@ -57,6 +57,8 @@ class Controller( object ):
client_files_default = os.path.join( self._db_dir, 'client_files' )
HydrusPaths.MakeSureDirectoryExists( self._server_files_dir )
HydrusPaths.MakeSureDirectoryExists( self._updates_dir )
HydrusPaths.MakeSureDirectoryExists( client_files_default )
HydrusGlobals.controller = self
@ -290,7 +292,9 @@ class Controller( object ):
def TidyUp( self ):
shutil.rmtree( self._db_dir )
time.sleep( 2 )
HydrusPaths.DeletePath( self._db_dir )
def ViewIsShutdown( self ):