RepoMirrors
/
hydrus
mirror of https://github.com/hydrusnetwork/hydrus.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Version 268

This commit is contained in:
Hydrus Network Developer 2017-08-09 16:33:51 -05:00
parent c6bee8395d
commit d55477e373
35 changed files with 1404 additions and 763 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

31
help/changelog.html
View File

@ -8,6 +8,37 @@
<div class="content">
<h3>changelog</h3>
<ul>
<li><h3>version 268</h3></li>
<ul>
<li>split the sort dropdown into two, splitting the sort type and sort order</li>
<li>file sorting works more intelligently behind the scenes in several ways</li>
<li>added 'sort by number of tags' to sort options</li>
<li>session pages now remember their sort status!</li>
<li>sessions also more reliably remember their actual thumbnail order for all pages (typically, this matters for importers atm)</li>
<li>network job controls now report an estimate of how long they will have to wait for bandwidth</li>
<li>subscriptions will now show some 'no more bandwidth to download files' text if they have to stop because of bandwidth rules</li>
<li>subscription 'should I start/continue' testing now has a little padding to forestall some potential unexpected long delays in operation due to edge cases</li>
<li>the edit subscription dialog now uses the new file import status control</li>
<li>fixed an issue with hentai foundry filters not applying (they added some categories since the downloader last worked, breaking the POST form), hence hiding most non-vanilla results</li>
<li>finished the 'database->migrate database' dialog and its help and removed the 'under construction' labels. the new help page is also now linked from the standard index.html</li>
<li>fixed an issue where pages would sometimes never initialise (due to being queued up after an infinite job like the network engine's mainloop!)</li>
<li>improved how all long-job threads are spawned</li>
<li>improved some more thread scheduling logic that meant some long-term jobs could be stacked undesirably</li>
<li>expanded how the new listctrl updates and deletes its data</li>
<li>reduced flicker on the new listctrl update events</li>
<li>the file import status window now uses the new listctrl</li>
<li>the manage subscriptions dialog now uses the new listctrl</li>
<li>collections now track their tags more accurately</li>
<li>moved the 'delete original files after success' checkbox up to reduce misclicks on ok'ing the file import dialog and added a bit of red warning text whenever it is on</li>
<li>sped up some behind-the-scenes content processing for large pages</li>
<li>improved how the program cleans some things up during exit</li>
<li>the issue with some clients not clearing their exit splash screen until a mouseover event should be fixed</li>
<li>at 120 open pages, the client will inform the user about the approaching max page limit</li>
<li>by default, import folders no longer delete anything. default is to ignore original files in all cases</li>
<li>deleted some old unusued code</li>
<li>misc cleanup</li>
<li>misc improvements</li>
</ul>
<li><h3>version 267</h3></li>
<ul>
<li>drag-and-dropping a 4chan or 8chan url onto the client will now automatically open a thread watcher for that url</li>

15
help/database_migration.html
View File

@ -6,7 +6,6 @@
</head>
<body>
<div class="content">
<p><b class="warning">DRAFT</b></p>
<h3>the hydrus database</h3>
<p>A hydrus client consists of three components:</p>
<ol>
@ -28,14 +27,16 @@
</ol>
<h3>these components can be put on different drives</h3>
<p>Although an initial install will keep these parts together, it is possible to run the database on a fast drive but keep your media in cheap slow storage. And if you have a very large collection, you can even spread your files across multiple drives. It is not very technically difficult, but I do not recommend it for new users.</p>
<p>Backing such an arrangement up is obviously more complicated, and the internal client backup is not sophisticated enough to capture everything, so I recommend you figure out a broader solution with a third-party backup program like FreeFileSync.</p>
<h3>pulling your media apart</h3>
<p><b class="warning">As always, I recommend creating a backup before you try any of this, just in case it goes wrong.</b></p>
<p>If you have multiple drives and would like to spread your media across them, please do not move the folders around yourself--the database has an internal 'knowledge' of where it thinks its file and thumbnail folders are, and if you move them while it is closed, it will throw 'missing path' errors as soon as it boots. The internal hydrus logic of relative and absolute paths is not always obvious, so it is easy to make mistakes, even if you think you know what you are doing. Instead, please do it through the gui:</p>
<p>Go <i>database->migration</i>, giving you this dialog:</p>
<p><img src="db_migration.png" /></p>
<p>This is from my main laptop client that I use day to day. I have moved the main database and its files out of the install directory but otherwise kept everything together. Your situation may be simpler or more complicated.</p>
<p><b>Portable</b> means that the path is beneath the main db dir and so is stored as a relative path. Portable paths will still function if the database changes location between boots (for instance, if you run the client from a USB drive and it mounts under a different location).</p>
<p><b>Weight</b> means the relative amount of media you would like to store in that location.</p>
<p>The operations on this dialog are simple and atomic--at no point is your db ever invalid. Once you have the locations and ideal usage set how you like, hit the 'move files now' button to actually shuffle your files around. It will take some time to finish.</p>
<p><b>Weight</b> means the relative amount of media you would like to store in that location. If location A has a weight of 1 and B has a weight of 2, A will get approximately one third of your files and B will get approximately two thirds.</p>
<p>The operations on this dialog are simple and atomic--at no point is your db ever invalid. Once you have the locations and ideal usage set how you like, hit the 'move files now' button to actually shuffle your files around. It will take some time to finish, but you can pause and resume it later if the job is large or you want to alter a path.</p>
<p>If you decide to move your database, the program will have to shut down first. Before you boot up again, you will have to create a new program shortcut:</p>
<h3>informing the software that the database has moved</h3>
<p>A straight call to the client executable will look for a database in <i>install_dir/db</i>. If one is not found, it will create one. So, if you move your database and then try to run the client again, it will try to create a new empty database in the previous location!</p>
@ -46,13 +47,13 @@
<li>client --db_dir="G:\misc documents\New Folder (3)\DO NOT ENTER"</li>
</ul>
<p>And it will instead use the given path. You can use any path that is valid in your system, but I would not advise using network locations and so on, as the database works best with some clever device locking calls these interfaces may not provide.</p>
<p>Rather than typing the path out in a terminal every time you want to launch your external database, create a new shortcut with the argument in. Something like this:</p>
<p>Rather than typing the path out in a terminal every time you want to launch your external database, create a new shortcut with the argument in. Something like this, which is from my main development computer and tests that a fresh default install will run an existing database ok:</p>
<p><img src="db_migration_shortcut.png" /></p>
<p>Note that an install with an 'external' database no longer needs access to write to its own path, so you can store it anywhere you like. If you move it, just double-check your shortcuts are still good and you are done.</p>
<p>Note that an install with an 'external' database no longer needs access to write to its own path, so you can store it anywhere you like (e.g. in 'Program Files'). If you move it, just double-check your shortcuts are still good and you are done.</p>
<h3>finally</h3>
<p>Now you have a new database in one or more new locations, make sure to update your backup routine to follow these new paths!</p>
<p>If your database now lives in one or more new locations, make sure to update your backup routine to follow them!</p>
<h3>p.s. running multiple clients</h3>
<p>Since you now know how to tell the software about an external database, you can, if you like, run multiple clients from the same install (and if you previously had multiple install folders, now you can now just use the one). Just make multiple shortcuts to the same client executable but with different database directories. They can run at the same time. You'll save yourself a little memory and update-hassle.</p>
<p>Since you now know how to tell the software about an external database, you can, if you like, run multiple clients from the same install (and if you previously had multiple install folders, now you can now just use the one). Just make multiple shortcuts to the same client executable but with different database directories. They can run at the same time. You'll save yourself a little memory and update-hassle. I do this on my laptop client to run a regular client for my media and a separate 'admin' client to do PTR petitions and so on.</p>
</div>
</body>
</html>

BIN
help/db_migration.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 26 KiB

1
help/index.html
View File

@ -41,6 +41,7 @@
<li><a href="advanced.html">advanced usage - general</a></li>
<li><a href="advanced_siblings.html">advanced usage - tag siblings</a></li>
<li><a href="advanced_parents.html">advanced usage - tag parents</a></li>
<li><a href="database_migration.html">database migration</a></li>
<li><a href="ipfs.html">ipfs</a></li>
<li><a href="local_booru.html">the local booru</a></li>
<li><a href="server.html">setting up your own server</a></li>

2
include/ClientCaches.py
View File

@ -1777,7 +1777,7 @@ class ThumbnailCache( object ):
self.Clear()
threading.Thread( target = self.DAEMONWaterfall, name = 'Waterfall Daemon' ).start()
self._controller.CallToThreadLongRunning( self.DAEMONWaterfall )
self._controller.sub( self, 'Clear', 'thumbnail_resize' )

75
include/ClientConstants.py
View File

@ -342,14 +342,6 @@ SHUTDOWN_TIMESTAMP_DELETE_ORPHANS = 2
( SizeChangedEvent, EVT_SIZE_CHANGED ) = wx.lib.newevent.NewCommandEvent()
SORT_BY_SMALLEST = 0
SORT_BY_LARGEST = 1
SORT_BY_SHORTEST = 2
SORT_BY_LONGEST = 3
SORT_BY_NEWEST = 4
SORT_BY_OLDEST = 5
SORT_BY_MIME = 6
SORT_BY_RANDOM = 7
SORT_BY_LEXICOGRAPHIC_ASC = 8
SORT_BY_LEXICOGRAPHIC_DESC = 9
SORT_BY_INCIDENCE_ASC = 10
@ -358,52 +350,33 @@ SORT_BY_LEXICOGRAPHIC_NAMESPACE_ASC = 12
SORT_BY_LEXICOGRAPHIC_NAMESPACE_DESC = 13
SORT_BY_INCIDENCE_NAMESPACE_ASC = 14
SORT_BY_INCIDENCE_NAMESPACE_DESC = 15
SORT_BY_WIDTH_ASC = 16
SORT_BY_WIDTH_DESC = 17
SORT_BY_HEIGHT_ASC = 18
SORT_BY_HEIGHT_DESC = 19
SORT_BY_RATIO_ASC = 20
SORT_BY_RATIO_DESC = 21
SORT_BY_NUM_PIXELS_ASC = 22
SORT_BY_NUM_PIXELS_DESC = 23
SORT_FILES_BY_FILESIZE = 0
SORT_FILES_BY_DURATION = 1
SORT_FILES_BY_IMPORT_TIME = 2
SORT_FILES_BY_MIME = 3
SORT_FILES_BY_RANDOM = 4
SORT_FILES_BY_WIDTH = 5
SORT_FILES_BY_HEIGHT = 6
SORT_FILES_BY_RATIO = 7
SORT_FILES_BY_NUM_PIXELS = 8
SORT_FILES_BY_NUM_TAGS = 9
SORT_ASC = 0
SORT_DESC = 1
SORT_CHOICES = []
SORT_CHOICES.append( ( 'system', SORT_BY_SMALLEST ) )
SORT_CHOICES.append( ( 'system', SORT_BY_LARGEST ) )
SORT_CHOICES.append( ( 'system', SORT_BY_SHORTEST ) )
SORT_CHOICES.append( ( 'system', SORT_BY_LONGEST ) )
SORT_CHOICES.append( ( 'system', SORT_BY_NEWEST ) )
SORT_CHOICES.append( ( 'system', SORT_BY_OLDEST ) )
SORT_CHOICES.append( ( 'system', SORT_BY_WIDTH_ASC ) )
SORT_CHOICES.append( ( 'system', SORT_BY_WIDTH_DESC ) )
SORT_CHOICES.append( ( 'system', SORT_BY_HEIGHT_ASC ) )
SORT_CHOICES.append( ( 'system', SORT_BY_HEIGHT_DESC ) )
SORT_CHOICES.append( ( 'system', SORT_BY_RATIO_DESC ) )
SORT_CHOICES.append( ( 'system', SORT_BY_RATIO_ASC ) )
SORT_CHOICES.append( ( 'system', SORT_BY_NUM_PIXELS_ASC ) )
SORT_CHOICES.append( ( 'system', SORT_BY_NUM_PIXELS_DESC ) )
SORT_CHOICES.append( ( 'system', SORT_BY_MIME ) )
SORT_CHOICES.append( ( 'system', SORT_BY_RANDOM ) )
sort_string_lookup = {}
sort_string_lookup[ SORT_BY_SMALLEST ] = 'smallest filesize first'
sort_string_lookup[ SORT_BY_LARGEST ] = 'largest filesize first'
sort_string_lookup[ SORT_BY_SHORTEST ] = 'shortest duration first'
sort_string_lookup[ SORT_BY_LONGEST ] = 'longest duration first'
sort_string_lookup[ SORT_BY_NEWEST ] = 'most recently imported first'
sort_string_lookup[ SORT_BY_OLDEST ] = 'least recently imported first'
sort_string_lookup[ SORT_BY_MIME ] = 'mime'
sort_string_lookup[ SORT_BY_RANDOM ] = 'random order'
sort_string_lookup[ SORT_BY_WIDTH_ASC ] = 'least wide first'
sort_string_lookup[ SORT_BY_WIDTH_DESC ] = 'most wide first'
sort_string_lookup[ SORT_BY_HEIGHT_ASC ] = 'least tall first'
sort_string_lookup[ SORT_BY_HEIGHT_DESC ] = 'most tall first'
sort_string_lookup[ SORT_BY_RATIO_ASC ] = 'tallest ratio first'
sort_string_lookup[ SORT_BY_RATIO_DESC ] = 'widest ratio first'
sort_string_lookup[ SORT_BY_NUM_PIXELS_ASC ] = 'fewest pixels first'
sort_string_lookup[ SORT_BY_NUM_PIXELS_DESC ] = 'most pixels first'
SORT_CHOICES.append( ( 'system', SORT_FILES_BY_FILESIZE ) )
SORT_CHOICES.append( ( 'system', SORT_FILES_BY_DURATION ) )
SORT_CHOICES.append( ( 'system', SORT_FILES_BY_IMPORT_TIME ) )
SORT_CHOICES.append( ( 'system', SORT_FILES_BY_MIME ) )
SORT_CHOICES.append( ( 'system', SORT_FILES_BY_RANDOM ) )
SORT_CHOICES.append( ( 'system', SORT_FILES_BY_WIDTH ) )
SORT_CHOICES.append( ( 'system', SORT_FILES_BY_HEIGHT ) )
SORT_CHOICES.append( ( 'system', SORT_FILES_BY_RATIO ) )
SORT_CHOICES.append( ( 'system', SORT_FILES_BY_NUM_PIXELS ) )
SORT_CHOICES.append( ( 'system', SORT_FILES_BY_NUM_TAGS ) )
STATUS_UNKNOWN = 0
STATUS_SUCCESSFUL = 1

26
include/ClientController.py
View File

@ -89,7 +89,7 @@ class Controller( HydrusController.HydrusController ):
if self._splash is not None:
self._splash.Destroy()
wx.CallAfter( self._splash.Destroy )
self._splash = None
@ -294,7 +294,10 @@ class Controller( HydrusController.HydrusController ):
while not image_renderer.IsReady():
if HydrusData.TimeHasPassed( start_time + 15 ): raise Exception( 'The image did not render in fifteen seconds, so the attempt to copy it to the clipboard was abandoned.' )
if HydrusData.TimeHasPassed( start_time + 15 ):
raise Exception( 'The image did not render in fifteen seconds, so the attempt to copy it to the clipboard was abandoned.' )
time.sleep( 0.1 )
@ -448,9 +451,7 @@ class Controller( HydrusController.HydrusController ):
exit_thread = threading.Thread( target = self.THREADExitEverything, name = 'Application Exit Thread' )
exit_thread.start()
self.CallToThreadLongRunning( self.THREADExitEverything )
except:
@ -581,7 +582,7 @@ class Controller( HydrusController.HydrusController ):
self.network_engine = ClientNetworking.NetworkEngine( self, bandwidth_manager, session_manager, login_manager )
self.CallToThread( self.network_engine.MainLoop )
self.CallToThreadLongRunning( self.network_engine.MainLoop )
#
@ -809,7 +810,7 @@ class Controller( HydrusController.HydrusController ):
disk_cache_stop_time = HydrusData.GetNow() + 1
HG.client_controller.Read( 'load_into_disk_cache', stop_time = disk_cache_stop_time, caller_limit = disk_cache_maintenance_mb * 1024 * 1024 )
self.Read( 'load_into_disk_cache', stop_time = disk_cache_stop_time, caller_limit = disk_cache_maintenance_mb * 1024 * 1024 )
@ -993,9 +994,7 @@ class Controller( HydrusController.HydrusController ):
HydrusData.RestartProcess()
restart_thread = threading.Thread( target = THREADRestart, name = 'Application Restart Thread' )
restart_thread.start()
self.CallToThreadLongRunning( THREADRestart )
@ -1015,9 +1014,7 @@ class Controller( HydrusController.HydrusController ):
self._CreateSplash()
boot_thread = threading.Thread( target = self.THREADBootEverything, name = 'Application Boot Thread' )
boot_thread.start()
self.CallToThreadLongRunning( self.THREADBootEverything )
self._app.MainLoop()
@ -1243,6 +1240,9 @@ class Controller( HydrusController.HydrusController ):
self.ShutdownModel()
self.pub( 'splash_set_title_text', u'cleaning up\u2026' )
self.pub( 'splash_set_status_text', u'' )
HydrusData.CleanRunningFile( self.db_dir, 'client' )
except HydrusExceptions.PermissionException:

2
include/ClientDB.py
View File

@ -7675,7 +7675,7 @@ class DB( HydrusDB.HydrusDB ):
self._controller.pub( 'splash_set_status_text', status, print_to_log = False )
job_key.SetVariable( 'popup_text_1', status )
stop_time = HydrusData.GetNow() + min( 5 + ( num_updates_to_do * 2 ), 30 )
stop_time = HydrusData.GetNow() + min( 5 + ( num_updates_to_do * 4 ), 30 )
self._LoadIntoDiskCache( stop_time = stop_time )

57
include/ClientData.py
View File

@ -388,21 +388,17 @@ def GetSearchURLs( url ):
return search_urls
def GetSortChoices( add_namespaces_and_ratings = True ):
def GetSortTypeChoices():
sort_choices = list( CC.SORT_CHOICES )
if add_namespaces_and_ratings:
sort_choices.extend( HC.options[ 'sort_by' ] )
service_keys = HG.client_controller.services_manager.GetServiceKeys( ( HC.LOCAL_RATING_LIKE, HC.LOCAL_RATING_NUMERICAL ) )
for service_key in service_keys:
sort_choices.extend( HC.options[ 'sort_by' ] )
service_keys = HG.client_controller.services_manager.GetServiceKeys( ( HC.LOCAL_RATING_LIKE, HC.LOCAL_RATING_NUMERICAL ) )
for service_key in service_keys:
sort_choices.append( ( 'rating_descend', service_key ) )
sort_choices.append( ( 'rating_ascend', service_key ) )
sort_choices.append( ( 'rating', service_key ) )
return sort_choices
@ -977,6 +973,13 @@ class ClientOptions( HydrusSerialisable.SerialisableBase ):
self._dictionary[ 'suggested_tags' ][ 'favourites' ] = {}
#
import ClientMedia
self._dictionary[ 'default_sort' ] = ClientMedia.MediaSort( ( 'system', CC.SORT_FILES_BY_FILESIZE ), CC.SORT_ASC )
self._dictionary[ 'fallback_sort' ] = ClientMedia.MediaSort( ( 'system', CC.SORT_FILES_BY_IMPORT_TIME ), CC.SORT_ASC )
def _InitialiseFromSerialisableInfo( self, serialisable_info ):
@ -1236,6 +1239,14 @@ class ClientOptions( HydrusSerialisable.SerialisableBase ):
def GetDefaultSort( self ):
with self._lock:
return self._dictionary[ 'default_sort' ]
def GetDuplicateActionOptions( self, duplicate_type ):
with self._lock:
@ -1244,6 +1255,14 @@ class ClientOptions( HydrusSerialisable.SerialisableBase ):
def GetFallbackSort( self ):
with self._lock:
return self._dictionary[ 'fallback_sort' ]
def GetFrameLocation( self, frame_key ):
with self._lock:
@ -1443,6 +1462,14 @@ class ClientOptions( HydrusSerialisable.SerialisableBase ):
def SetDefaultSort( self, media_sort ):
with self._lock:
self._dictionary[ 'default_sort' ] = media_sort
def SetDuplicateActionOptions( self, duplicate_type, duplicate_action_options ):
with self._lock:
@ -1451,6 +1478,14 @@ class ClientOptions( HydrusSerialisable.SerialisableBase ):
def SetFallbackSort( self, media_sort ):
with self._lock:
self._dictionary[ 'fallback_sort' ] = media_sort
def SetFrameLocation( self, frame_key, remember_size, remember_position, last_size, last_position, default_gravity, default_position, maximised, fullscreen ):
with self._lock:

41
include/ClientDefaults.py
View File

@ -91,8 +91,6 @@ def GetClientDefaultOptions():
options = {}
options[ 'play_dumper_noises' ] = True
options[ 'default_sort' ] = 0 # smallest
options[ 'sort_fallback' ] = 4 # newest
options[ 'default_collect' ] = None
options[ 'export_path' ] = None
options[ 'hpos' ] = 400
@ -214,27 +212,32 @@ def GetDefaultHentaiFoundryInfo():
info = {}
info[ 'rating_nudity' ] = 3
info[ 'rating_violence' ] = 3
info[ 'rating_profanity' ] = 3
info[ 'rating_racism' ] = 3
info[ 'rating_sex' ] = 3
info[ 'rating_spoilers' ] = 3
info[ 'rating_nudity' ] = '3'
info[ 'rating_violence' ] = '3'
info[ 'rating_profanity' ] = '3'
info[ 'rating_racism' ] = '3'
info[ 'rating_sex' ] = '3'
info[ 'rating_spoilers' ] = '3'
info[ 'rating_yaoi' ] = 1
info[ 'rating_yuri' ] = 1
info[ 'rating_teen' ] = 1
info[ 'rating_guro' ] = 1
info[ 'rating_furry' ] = 1
info[ 'rating_beast' ] = 1
info[ 'rating_male' ] = 1
info[ 'rating_female' ] = 1
info[ 'rating_futa' ] = 1
info[ 'rating_other' ] = 1
info[ 'rating_yaoi' ] = '1'
info[ 'rating_yuri' ] = '1'
info[ 'rating_teen' ] = '1'
info[ 'rating_guro' ] = '1'
info[ 'rating_furry' ] = '1'
info[ 'rating_beast' ] = '1'
info[ 'rating_male' ] = '1'
info[ 'rating_female' ] = '1'
info[ 'rating_futa' ] = '1'
info[ 'rating_other' ] = '1'
info[ 'rating_scat' ] = '1'
info[ 'rating_incest' ] = '1'
info[ 'rating_rape' ] = '1'
info[ 'filter_media' ] = 'A'
info[ 'filter_order' ] = 'date_new'
info[ 'filter_type' ] = 0
info[ 'filter_type' ] = '0'
info[ 'yt0' ] = 'Apply' # the submit button wew lad
return info

2
include/ClientDownloading.py
View File

@ -408,7 +408,7 @@ def ParseImageboardThreadURL( thread_url ):
except Exception as e:
raise Exception( 'Could not understand the board or thread id!' )
raise Exception( 'Could not understand that thread url! Either the board or the thread id components were malformed or missing.' )
return ( thread_url, host, board, thread_id )

132
include/ClientGUI.py
View File

@ -1297,7 +1297,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
ClientGUIMenus.AppendSeparator( menu )
ClientGUIMenus.AppendMenuItem( self, menu, 'migrate database (under construction!)', 'Review and manage the locations your database is stored.', self._MigrateDatabase )
ClientGUIMenus.AppendMenuItem( self, menu, 'migrate database', 'Review and manage the locations your database is stored.', self._MigrateDatabase )
ClientGUIMenus.AppendSeparator( menu )
@ -1655,6 +1655,38 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
return insertion_index
def _GetCurrentMediaPage( self ):
page = self._notebook.GetCurrentPage()
if page is not None:
while isinstance( page, wx.Notebook ):
page = page.GetCurrentPage()
return page
def _GetMediaPages( self ):
results = []
for page in self._GetPages():
if isinstance( page, wx.Notebook ):
results.extend( page.GetMediaPages() )
else:
results.append( page )
def _GetPageAndIndex( self, page_key ):
for ( page, index ) in ( ( self._notebook.GetPage( index ), index ) for index in range( self._notebook.GetPageCount() ) ):
@ -1668,6 +1700,11 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
raise HydrusExceptions.DataMissing()
def _GetPages( self ):
return [ self._notebook.GetPage( i ) for i in range( self._notebook.GetPageCount() ) ]
def _ImportFiles( self, paths = None ):
if paths is None: paths = []
@ -1863,7 +1900,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
def _LoadGUISession( self, name ):
for page in [ self._notebook.GetPage( i ) for i in range( self._notebook.GetPageCount() ) ]:
for page in self._GetPages():
try:
@ -2138,6 +2175,11 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
return
if self._notebook.GetPageCount() == 120:
HydrusData.ShowText( 'You have 120 pages open! You can only open a few more before system stability is affected! Please close some now!' )
self._controller.ResetIdleTimer()
self._controller.ResetPageChangeTimer()
@ -2396,7 +2438,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
def _Refresh( self ):
page = self._notebook.GetCurrentPage()
page = self._GetCurrentMediaPage()
if page is not None:
@ -2412,7 +2454,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
else:
page = self._notebook.GetCurrentPage()
page = self._GetCurrentMediaPage()
if page is None:
@ -2607,9 +2649,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
session = ClientGUIPages.GUISession( name )
for i in range( self._notebook.GetPageCount() ):
page = self._notebook.GetPage( i )
for page in self._GetPages():
management_controller = page.GetManagementController()
@ -2648,23 +2688,32 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
def _SetMediaFocus( self ):
page = self._notebook.GetCurrentPage()
page = self._GetCurrentMediaPage()
if page is not None: page.SetMediaFocus()
if page is not None:
page.SetMediaFocus()
def _SetSearchFocus( self ):
page = self._notebook.GetCurrentPage()
page = self._GetCurrentMediaPage()
if page is not None: page.SetSearchFocus()
if page is not None:
page.SetSearchFocus()
def _SetSynchronisedWait( self ):
page = self._notebook.GetCurrentPage()
page = self._GetCurrentMediaPage()
if page is not None: page.SetSynchronisedWait()
if page is not None:
page.SetSynchronisedWait()
def _SetupBackupPath( self ):
@ -2749,11 +2798,10 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
def _ShowHideSplitters( self ):
page = self._notebook.GetCurrentPage()
page = self._GetCurrentMediaPage()
if page is not None:
@ -2761,6 +2809,27 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
def _ShowPage( self, showee ):
for ( i, page ) in enumerate( self._GetPages() ):
if isinstance( page, wx.Notebook ) and page.HasPage( page ):
self._notebook.SetSelection( i )
page.ShowPage( page )
break
elif page == showee:
self._notebook.SetSelection( i )
break
def _StartIPFSDownload( self ):
ipfs_services = self._controller.services_manager.GetServices( ( HC.IPFS, ) )
@ -3270,7 +3339,7 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
def EventFocus( self, event ):
page = self._notebook.GetCurrentPage()
page = self._GetCurrentMediaPage()
if page is not None:
@ -3467,7 +3536,7 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
try:
for page in [ self._notebook.GetPage( i ) for i in range( self._notebook.GetPageCount() ) ]:
for page in self._GetPages():
page.TestAbleToClose()
@ -3491,9 +3560,12 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
self._message_manager.Hide()
for page in [ self._notebook.GetPage( i ) for i in range( self._notebook.GetPageCount() ) ]: page.CleanBeforeDestroy()
for page in self._GetPages():
page.CleanBeforeDestroy()
page = self._notebook.GetCurrentPage()
page = self._GetCurrentMediaPage()
if page is not None:
@ -3566,7 +3638,7 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
def GetCurrentPage( self ):
return self._notebook.GetCurrentPage()
return self._GetCurrentMediaPage()
def IShouldRegularlyUpdate( self, window ):
@ -3605,19 +3677,16 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
else:
if True not in ( page.IsURLImportPage() for page in [ self._notebook.GetPage( i ) for i in range( self._notebook.GetPageCount() ) ] ):
if True not in ( page.IsURLImportPage() for page in self._GetMediaPages() ):
self._NewPageImportURLs()
for ( page, i ) in [ ( self._notebook.GetPage( i ), i ) for i in range( self._notebook.GetPageCount() ) ]:
for page in self._GetMediaPages():
if page.IsURLImportPage():
if page != self._notebook.GetCurrentPage():
self._notebook.SetSelection( i )
self._ShowPage( page )
page_key = page.GetPageKey()
@ -3631,7 +3700,7 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
def IsCurrentPage( self, page_key ):
result = self._notebook.GetCurrentPage()
result = self._GetCurrentMediaPage()
if result is None:
@ -4021,7 +4090,10 @@ class FrameSplash( wx.Frame ):
event.Skip()
def EventEraseBackground( self, event ): pass
def EventEraseBackground( self, event ):
pass
def EventPaint( self, event ):
@ -4044,7 +4116,7 @@ class FrameSplash( wx.Frame ):
self._dirty = True
self.Refresh()
wx.CallAfter( self.Refresh )
def SetTitleText( self, text, print_to_log = True ):
@ -4058,6 +4130,6 @@ class FrameSplash( wx.Frame ):
self._dirty = True
self.Refresh()
wx.CallAfter( self.Refresh )

198
include/ClientGUICommon.py
View File

@ -3,6 +3,7 @@ import ClientData
import ClientConstants as CC
import ClientGUIMenus
import ClientGUITopLevelWindows
import ClientMedia
import ClientRatings
import ClientThreading
import HydrusConstants as HC
@ -422,12 +423,16 @@ class CheckboxCollect( wx.combo.ComboCtrl ):
self._page_key = page_key
popup = self._Popup()
self._collect_by = HC.options[ 'default_collect' ]
popup = self._Popup( self._collect_by )
#self.UseAltPopupWindow( True )
self.SetPopupControl( popup )
self.SetValue( 'no collections' )
def GetChoice( self ):
@ -445,14 +450,16 @@ class CheckboxCollect( wx.combo.ComboCtrl ):
class _Popup( wx.combo.ComboPopup ):
def __init__( self ):
def __init__( self, collect_by ):
wx.combo.ComboPopup.__init__( self )
self._initial_collect_by = collect_by
def Create( self, parent ):
self._control = self._Control( parent, self.GetCombo() )
self._control = self._Control( parent, self.GetCombo(), self._initial_collect_by )
return True
@ -469,7 +476,7 @@ class CheckboxCollect( wx.combo.ComboCtrl ):
class _Control( wx.CheckListBox ):
def __init__( self, parent, special_parent ):
def __init__( self, parent, special_parent, collect_by ):
text_and_data_tuples = set()
@ -503,14 +510,12 @@ class CheckboxCollect( wx.combo.ComboCtrl ):
self._special_parent = special_parent
default = HC.options[ 'default_collect' ]
self.SetValue( default )
self.Bind( wx.EVT_CHECKLISTBOX, self.EventChanged )
self.Bind( wx.EVT_LEFT_DOWN, self.EventLeftDown )
wx.CallAfter( self.SetValue, collect_by )
def _BroadcastCollect( self ):
@ -584,78 +589,140 @@ class CheckboxCollect( wx.combo.ComboCtrl ):
self.SetChecked( indices_to_check )
self._BroadcastCollect()
if len( indices_to_check ) > 0:
self.SetChecked( indices_to_check )
self._BroadcastCollect()
class ChoiceSort( BetterChoice ):
class ChoiceSort( wx.Panel ):
def __init__( self, parent, page_key = None, add_namespaces_and_ratings = True ):
def __init__( self, parent, management_controller = None ):
BetterChoice.__init__( self, parent )
wx.Panel.__init__( self, parent )
self._page_key = page_key
self._management_controller = management_controller
services_manager = HG.client_controller.services_manager
self._sort_type_choice = BetterChoice( self )
self._sort_asc_choice = BetterChoice( self )
sort_choices = ClientData.GetSortChoices( add_namespaces_and_ratings = add_namespaces_and_ratings )
asc_width = ClientData.ConvertTextToPixelWidth( self._sort_asc_choice, 15 )
for sort_by in sort_choices:
self._sort_asc_choice.SetMinSize( ( asc_width, -1 ) )
sort_types = ClientData.GetSortTypeChoices()
for sort_type in sort_types:
( sort_by_type, sort_by_data ) = sort_by
example_sort = ClientMedia.MediaSort( sort_type, CC.SORT_ASC )
if sort_by_type == 'system':
label = CC.sort_string_lookup[ sort_by_data ]
elif sort_by_type == 'namespaces':
label = '-'.join( sort_by_data )
elif sort_by_type in ( 'rating_descend', 'rating_ascend' ):
service_key = sort_by_data
service = services_manager.GetService( service_key )
if sort_by_type == 'rating_descend':
label = service.GetName() + ' rating highest first'
elif sort_by_type == 'rating_ascend':
label = service.GetName() + ' rating lowest first'
self.Append( 'sort by ' + label, sort_by )
self._sort_type_choice.Append( example_sort.GetSortTypeString(), sort_type )
self.Bind( wx.EVT_CHOICE, self.EventChoice )
type_width = ClientData.ConvertTextToPixelWidth( self._sort_type_choice, 10 )
self._sort_type_choice.SetMinSize( ( type_width, -1 ) )
self._sort_asc_choice.Append( '', CC.SORT_ASC )
self._UpdateAscLabels()
#
hbox = wx.BoxSizer( wx.HORIZONTAL )
hbox.AddF( self._sort_type_choice, CC.FLAGS_EXPAND_BOTH_WAYS )
hbox.AddF( self._sort_asc_choice, CC.FLAGS_VCENTER )
self.SetSizer( hbox )
self._sort_type_choice.Bind( wx.EVT_CHOICE, self.EventSortTypeChoice )
self._sort_asc_choice.Bind( wx.EVT_CHOICE, self.EventSortAscChoice )
HG.client_controller.sub( self, 'ACollectHappened', 'collect_media' )
if self._management_controller is not None and self._management_controller.HasVariable( 'media_sort' ):
media_sort = self._management_controller.GetVariable( 'media_sort' )
try:
self.SetSort( media_sort )
except:
default_sort = ClientMedia.MediaSort( ( 'system', CC.SORT_FILES_BY_FILESIZE ), CC.SORT_ASC )
self.SetSort( default_sort )
def _BroadcastSort( self ):
selection = self.GetSelection()
media_sort = self._GetCurrentSort()
if selection != wx.NOT_FOUND:
if self._management_controller is not None:
sort_by = self.GetClientData( selection )
self._management_controller.SetVariable( 'media_sort', media_sort )
HG.client_controller.pub( 'sort_media', self._page_key, sort_by )
page_key = self._management_controller.GetKey( 'page' )
HG.client_controller.pub( 'sort_media', page_key, media_sort )
def _GetCurrentSort( self ):
sort_type = self._sort_type_choice.GetChoice()
sort_asc = self._sort_asc_choice.GetChoice()
media_sort = ClientMedia.MediaSort( sort_type, sort_asc )
return media_sort
def _UpdateAscLabels( self ):
media_sort = self._GetCurrentSort()
self._sort_asc_choice.Clear()
if media_sort.CanAsc():
( asc_str, desc_str ) = media_sort.GetSortAscStrings()
self._sort_asc_choice.Append( asc_str, CC.SORT_ASC )
self._sort_asc_choice.Append( desc_str, CC.SORT_DESC )
self._sort_asc_choice.SelectClientData( media_sort.sort_asc )
self._sort_asc_choice.Enable()
else:
self._sort_asc_choice.Append( '', CC.SORT_ASC )
self._sort_asc_choice.SelectClientData( CC.SORT_ASC )
self._sort_asc_choice.Disable()
def ACollectHappened( self, page_key, collect_by ):
if page_key == self._page_key:
if self._management_controller is not None:
self._BroadcastSort()
my_page_key = self._management_controller.GetKey( 'page' )
if page_key == my_page_key:
self._BroadcastSort()
@ -664,12 +731,29 @@ class ChoiceSort( BetterChoice ):
self._BroadcastSort()
def EventChoice( self, event ):
def EventSortAscChoice( self, event ):
if self._page_key is not None:
self._BroadcastSort()
self._BroadcastSort()
def EventSortTypeChoice( self, event ):
self._UpdateAscLabels()
self._BroadcastSort()
def GetSort( self ):
return self._GetCurrentSort()
def SetSort( self, media_sort ):
self._sort_type_choice.SelectClientData( media_sort.sort_type )
self._sort_asc_choice.SelectClientData( media_sort.sort_asc )
self._UpdateAscLabels()
class ExportPatternButton( wx.Button ):
@ -828,6 +912,8 @@ class Gauge( wx.Gauge ):
value = min( int( 1000 * ( float( value ) / self._actual_range ) ), 1000 )
value = min( value, self.GetRange() )
if value != self.GetValue():
wx.Gauge.SetValue( self, value )

24
include/ClientGUIDialogs.py
View File

@ -852,7 +852,11 @@ class DialogInputLocalFiles( Dialog ):
self._import_file_options = ClientGUICollapsible.CollapsibleOptionsImportFiles( self )
self._delete_after_success_st = ClientGUICommon.BetterStaticText( self, style = wx.ALIGN_RIGHT | wx.ST_NO_AUTORESIZE )
self._delete_after_success_st.SetForegroundColour( ( 127, 0, 0 ) )
self._delete_after_success = wx.CheckBox( self, label = 'delete original files after successful import' )
self._delete_after_success.Bind( wx.EVT_CHECKBOX, self.EventDeleteAfterSuccessCheck )
self._add_button = wx.Button( self, label = 'import now' )
self._add_button.Bind( wx.EVT_BUTTON, self.EventOK )
@ -872,6 +876,11 @@ class DialogInputLocalFiles( Dialog ):
gauge_sizer.AddF( self._progress_pause, CC.FLAGS_VCENTER )
gauge_sizer.AddF( self._progress_cancel, CC.FLAGS_VCENTER )
delete_hbox = wx.BoxSizer( wx.HORIZONTAL )
delete_hbox.AddF( self._delete_after_success_st, CC.FLAGS_EXPAND_BOTH_WAYS )
delete_hbox.AddF( self._delete_after_success, CC.FLAGS_VCENTER )
buttons = wx.BoxSizer( wx.HORIZONTAL )
buttons.AddF( self._add_button, CC.FLAGS_VCENTER )
@ -882,9 +891,8 @@ class DialogInputLocalFiles( Dialog ):
vbox.AddF( listctrl_panel, CC.FLAGS_EXPAND_BOTH_WAYS )
vbox.AddF( gauge_sizer, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
vbox.AddF( delete_hbox, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
vbox.AddF( self._import_file_options, CC.FLAGS_EXPAND_PERPENDICULAR )
vbox.AddF( self._delete_after_success, CC.FLAGS_LONE_BUTTON )
vbox.AddF( ( 0, 5 ), CC.FLAGS_NONE )
vbox.AddF( buttons, CC.FLAGS_BUTTON_SIZER )
self.SetSizer( vbox )
@ -1024,6 +1032,18 @@ class DialogInputLocalFiles( Dialog ):
self.EndModal( wx.ID_CANCEL )
def EventDeleteAfterSuccessCheck( self, event ):
if self._delete_after_success.GetValue():
self._delete_after_success_st.SetLabelText( 'YOUR ORIGINAL FILES WILL BE DELETED' )
else:
self._delete_after_success_st.SetLabelText( '' )
def EventOK( self, event ):
self._TidyUp()

82
include/ClientGUIListCtrl.py
View File

@ -551,7 +551,7 @@ class BetterListCtrl( wx.ListCtrl, ListCtrlAutoWidthMixin ):
( data, display_tuple, sort_tuple ) = data_info
index = wx.ListCtrl.Append( self, display_tuple )
index = self.Append( display_tuple )
self._indices_to_data_info[ index ] = data_info
self._data_to_indices[ data ] = index
@ -578,6 +578,22 @@ class BetterListCtrl( wx.ListCtrl, ListCtrlAutoWidthMixin ):
return indices
def _RecalculateIndicesAfterDelete( self ):
sorted_data_info = self._SortDataInfo()
self._indices_to_data_info = {}
self._data_to_indices = {}
for ( index, data_info ) in enumerate( sorted_data_info ):
( data, display_tuple, sort_tuple ) = data_info
self._data_to_indices[ data ] = index
self._indices_to_data_info[ index ] = data_info
def _SortDataInfo( self ):
data_infos = list( self._indices_to_data_info.values() )
@ -586,7 +602,7 @@ class BetterListCtrl( wx.ListCtrl, ListCtrlAutoWidthMixin ):
( data, display_tuple, sort_tuple ) = data_info
return sort_tuple[ self._sort_column ]
return ( sort_tuple[ self._sort_column ], sort_tuple ) # add the sort tuple to get secondary sorting
data_infos.sort( key = sort_key, reverse = not self._sort_asc )
@ -594,7 +610,7 @@ class BetterListCtrl( wx.ListCtrl, ListCtrlAutoWidthMixin ):
return data_infos
def _SortAndUpdate( self ):
def _SortAndRefreshRows( self ):
scroll_pos = self.GetScrollPos( wx.VERTICAL )
@ -641,7 +657,7 @@ class BetterListCtrl( wx.ListCtrl, ListCtrlAutoWidthMixin ):
del self._indices_to_data_info[ index ]
self._SortAndUpdate()
self._RecalculateIndicesAfterDelete()
def DeleteSelected( self ):
@ -661,7 +677,7 @@ class BetterListCtrl( wx.ListCtrl, ListCtrlAutoWidthMixin ):
del self._indices_to_data_info[ index ]
self._SortAndUpdate()
self._RecalculateIndicesAfterDelete()
def EventBeginColDrag( self, event ):
@ -702,7 +718,7 @@ class BetterListCtrl( wx.ListCtrl, ListCtrlAutoWidthMixin ):
self._sort_asc = True
self._SortAndUpdate()
self._SortAndRefreshRows()
def EventItemActivated( self, event ):
@ -785,21 +801,32 @@ class BetterListCtrl( wx.ListCtrl, ListCtrlAutoWidthMixin ):
def SetData( self, datas ):
old_selected_datas = self.GetData( only_selected = True )
datas = set( datas )
existing_datas = set( self._data_to_indices.keys() )
self._data_to_indices = {}
self._indices_to_data_info = {}
datas_to_add = datas.difference( existing_datas )
datas_to_update = datas.intersection( existing_datas )
datas_to_delete = existing_datas.difference( datas )
self.DeleteAllItems()
for data in datas:
if len( datas_to_delete ) > 0:
select = data in old_selected_datas
self.AddData( data, select = select )
self.DeleteDatas( datas_to_delete )
self._SortAndUpdate()
if len( datas_to_update ) > 0:
self.UpdateDatas( datas_to_update )
if len( datas_to_add ) > 0:
for data in datas_to_add:
self.AddData( data )
self._SortAndRefreshRows()
def Sort( self, col = None, asc = None ):
@ -814,6 +841,27 @@ class BetterListCtrl( wx.ListCtrl, ListCtrlAutoWidthMixin ):
self._sort_asc = asc
self._SortAndUpdate()
self._SortAndRefreshRows()
def UpdateDatas( self, datas ):
for data in datas:
( display_tuple, sort_tuple ) = self._data_to_tuples_func( data )
data_info = ( data, display_tuple, sort_tuple )
index = self._data_to_indices[ data ]
if data_info != self._indices_to_data_info[ index ]:
self._indices_to_data_info[ index ] = data_info
for ( column_index, value ) in enumerate( display_tuple ):
self.SetStringItem( index, column_index, value )

45
include/ClientGUIManagement.py
View File

@ -68,14 +68,13 @@ def CreateManagementController( page_name, management_type, file_service_key = N
file_service_key = CC.COMBINED_LOCAL_FILE_SERVICE_KEY
management_controller = ManagementController( page_name )
new_options = HG.client_controller.GetNewOptions()
# sort
# collect
# nah, these are only valid for types with regular file lists
management_controller = ManagementController( page_name )
management_controller.SetType( management_type )
management_controller.SetKey( 'file_service', file_service_key )
management_controller.SetVariable( 'media_sort', new_options.GetDefaultSort() )
return management_controller
@ -577,6 +576,8 @@ class ManagementController( HydrusSerialisable.SerialisableBase ):
def _InitialiseDefaults( self ):
self._serialisables[ 'media_sort' ] = ClientMedia.MediaSort( ( 'system', CC.SORT_FILES_BY_FILESIZE ), CC.SORT_ASC )
if self._management_type == MANAGEMENT_TYPE_DUPLICATE_FILTER:
self._keys[ 'duplicate_filter_file_domain' ] = CC.LOCAL_FILE_SERVICE_KEY
@ -651,7 +652,7 @@ class ManagementController( HydrusSerialisable.SerialisableBase ):
new_serialisable_info = ( page_name, management_type, serialisable_keys, serialisable_simples, serialisable_serialisables )
return( 3, new_serialisable_info )
return ( 3, new_serialisable_info )
@ -682,6 +683,11 @@ class ManagementController( HydrusSerialisable.SerialisableBase ):
def HasVariable( self, name ):
return name in self._simples or name in self._serialisables
def IsImporter( self ):
return self._management_type in ( MANAGEMENT_TYPE_IMPORT_GALLERY, MANAGEMENT_TYPE_IMPORT_HDD, MANAGEMENT_TYPE_IMPORT_PAGE_OF_IMAGES, MANAGEMENT_TYPE_IMPORT_THREAD_WATCHER, MANAGEMENT_TYPE_IMPORT_URLS )
@ -734,16 +740,7 @@ class ManagementPanel( wx.lib.scrolledpanel.ScrolledPanel ):
self._page = page
self._page_key = self._management_controller.GetKey( 'page' )
self._sort_by = ClientGUICommon.ChoiceSort( self, self._page_key )
try:
self._sort_by.SetSelection( HC.options[ 'default_sort' ] )
except:
self._sort_by.SetSelection( 0 )
self._sort_by = ClientGUICommon.ChoiceSort( self, management_controller = self._management_controller )
self._collect_by = ClientGUICommon.CheckboxCollect( self, self._page_key )
@ -1469,7 +1466,7 @@ class ManagementPanelImporterGallery( ManagementPanelImporter ):
vbox = wx.BoxSizer( wx.VERTICAL )
vbox.AddF( self._sort_by, CC.FLAGS_EXPAND_PERPENDICULAR )
vbox.AddF( self._sort_by, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
self._collect_by.Hide()
@ -1794,7 +1791,7 @@ class ManagementPanelImporterHDD( ManagementPanelImporter ):
vbox = wx.BoxSizer( wx.VERTICAL )
vbox.AddF( self._sort_by, CC.FLAGS_EXPAND_PERPENDICULAR )
vbox.AddF( self._sort_by, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
self._collect_by.Hide()
@ -1973,7 +1970,7 @@ class ManagementPanelImporterPageOfImages( ManagementPanelImporter ):
vbox = wx.BoxSizer( wx.VERTICAL )
vbox.AddF( self._sort_by, CC.FLAGS_EXPAND_PERPENDICULAR )
vbox.AddF( self._sort_by, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
self._collect_by.Hide()
@ -2313,7 +2310,7 @@ class ManagementPanelImporterThreadWatcher( ManagementPanelImporter ):
vbox = wx.BoxSizer( wx.VERTICAL )
vbox.AddF( self._sort_by, CC.FLAGS_EXPAND_PERPENDICULAR )
vbox.AddF( self._sort_by, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
self._collect_by.Hide()
@ -2599,7 +2596,7 @@ class ManagementPanelImporterURLs( ManagementPanelImporter ):
vbox = wx.BoxSizer( wx.VERTICAL )
vbox.AddF( self._sort_by, CC.FLAGS_EXPAND_PERPENDICULAR )
vbox.AddF( self._sort_by, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
self._collect_by.Hide()
@ -2910,7 +2907,7 @@ class ManagementPanelPetitions( ManagementPanel ):
vbox = wx.BoxSizer( wx.VERTICAL )
vbox.AddF( self._sort_by, CC.FLAGS_EXPAND_PERPENDICULAR )
vbox.AddF( self._sort_by, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
vbox.AddF( self._collect_by, CC.FLAGS_EXPAND_PERPENDICULAR )
vbox.AddF( self._petitions_info_panel, CC.FLAGS_EXPAND_PERPENDICULAR )
@ -3156,7 +3153,7 @@ class ManagementPanelPetitions( ManagementPanel ):
panel.Collect( self._page_key, self._collect_by.GetChoice() )
panel.Sort( self._page_key, self._sort_by.GetChoice() )
panel.Sort( self._page_key, self._sort_by.GetSort() )
self._controller.pub( 'swap_media_panel', self._page_key, panel )
@ -3333,7 +3330,7 @@ class ManagementPanelQuery( ManagementPanel ):
vbox = wx.BoxSizer( wx.VERTICAL )
vbox.AddF( self._sort_by, CC.FLAGS_EXPAND_PERPENDICULAR )
vbox.AddF( self._sort_by, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
vbox.AddF( self._collect_by, CC.FLAGS_EXPAND_PERPENDICULAR )
if self._search_enabled: vbox.AddF( self._search_panel, CC.FLAGS_EXPAND_PERPENDICULAR )
@ -3486,7 +3483,7 @@ class ManagementPanelQuery( ManagementPanel ):
panel.Collect( self._page_key, self._collect_by.GetChoice() )
panel.Sort( self._page_key, self._sort_by.GetChoice() )
panel.Sort( self._page_key, self._sort_by.GetSort() )
self._controller.pub( 'swap_media_panel', self._page_key, panel )

8
include/ClientGUIMedia.py
View File

@ -1649,11 +1649,11 @@ class MediaPanel( ClientMedia.ListeningMediaList, wx.ScrolledWindow ):
pass
def Sort( self, page_key, sort_by = None ):
def Sort( self, page_key, media_sort = None ):
if page_key == self._page_key:
ClientMedia.ListeningMediaList.Sort( self, sort_by )
ClientMedia.ListeningMediaList.Sort( self, media_sort )
@ -3540,9 +3540,9 @@ class MediaPanelThumbnails( MediaPanel ):
def Sort( self, page_key, sort_by = None ):
def Sort( self, page_key, media_sort = None ):
MediaPanel.Sort( self, page_key, sort_by )
MediaPanel.Sort( self, page_key, media_sort )
self._DirtyAllPages()

114
include/ClientGUIPages.py
View File

@ -329,6 +329,120 @@ class Page( wx.SplitterWindow ):
wx.CallAfter( self.SetMediaResults, sorted_initial_media_results )
class PagesNotebook( wx.Notebook ):
def __init__( self, parent, controller ):
# bring the gui's current _notebook into here and merge all the rename_page, new_page, and other stuff into this
wx.Notebook.__init__( self, parent )
self._controller = controller
def _GetMediaPages( self ):
results = []
for page in self._GetPages():
if isinstance( page, wx.Notebook ):
results.extend( page.GetMediaPages() )
else:
results.append( page )
return results
def _GetPages( self ):
return [ self.GetPage( i ) for i in range( self.GetPageCount() ) ]
def CleanBeforeDestroy( self ):
for page in self._GetPages():
page.CleanBeforeDestroy()
def GetNumFiles( self ):
return sum( page.GetNumFiles() for page in self._GetPages() )
def GetMediaPages( self ):
return self._GetMediaPages()
def GetPages( self ):
return self._GetPages()
def GetPrettyStatus( self ):
current_page = self.GetCurrentPage()
if current_page is None:
return ''
else:
return current_page.GetPrettyStatus()
def HasPage( self, page ):
return page in self._GetMediaPages()
def PrepareToHide( self ):
for page in self._GetPages():
page.PrepareToHide()
def ShowPage( self, showee ):
for ( i, page ) in enumerate( self._GetPages() ):
if isinstance( page, wx.Notebook ) and page.HasPage( showee ):
self.SetSelection( i )
page.ShowPage( showee )
break
elif page == showee:
self.SetSelection( i )
break
def TestAbleToClose( self ):
for page in self._GetPages():
page.TestAbleToClose()
class GUISession( HydrusSerialisable.SerialisableBaseNamed ):
SERIALISABLE_TYPE = HydrusSerialisable.SERIALISABLE_TYPE_GUI_SESSION

23
include/ClientGUIScrolledPanelsEdit.py
View File

@ -12,6 +12,7 @@ import ClientGUIListBoxes
import ClientGUIListCtrl
import ClientGUIMenus
import ClientGUIScrolledPanels
import ClientGUISeedCache
import ClientGUITopLevelWindows
import HydrusConstants as HC
import HydrusData
@ -1198,7 +1199,7 @@ class EditSubscriptionPanel( ClientGUIScrolledPanels.EditPanel ):
self._last_checked_st = wx.StaticText( self._info_panel )
self._next_check_st = wx.StaticText( self._info_panel )
self._seed_info_st = wx.StaticText( self._info_panel )
self._seed_cache_control = ClientGUISeedCache.SeedCacheStatusControl( self._info_panel, HG.client_controller )
#
@ -1264,9 +1265,6 @@ class EditSubscriptionPanel( ClientGUIScrolledPanels.EditPanel ):
self._paused = wx.CheckBox( self._control_panel )
self._seed_cache_button = ClientGUICommon.BetterBitmapButton( self._control_panel, CC.GlobalBMPs.seed_cache, self._SeedCache )
self._seed_cache_button.SetToolTipString( 'open detailed url cache status' )
self._retry_failed = ClientGUICommon.BetterButton( self._control_panel, 'retry failed', self.RetryFailed )
self._check_now_button = ClientGUICommon.BetterButton( self._control_panel, 'force check on dialog ok', self.CheckNow )
@ -1328,6 +1326,8 @@ class EditSubscriptionPanel( ClientGUIScrolledPanels.EditPanel ):
self._check_now_button.Disable()
self._seed_cache = self._seed_cache.Duplicate() # so that if it is edited but we cancel, this doesn't screw up
self._UpdateCommandButtons()
self._UpdateLastNextCheck()
self._UpdateSeedInfo()
@ -1336,7 +1336,7 @@ class EditSubscriptionPanel( ClientGUIScrolledPanels.EditPanel ):
self._info_panel.AddF( self._last_checked_st, CC.FLAGS_EXPAND_PERPENDICULAR )
self._info_panel.AddF( self._next_check_st, CC.FLAGS_EXPAND_PERPENDICULAR )
self._info_panel.AddF( self._seed_info_st, CC.FLAGS_EXPAND_PERPENDICULAR )
self._info_panel.AddF( self._seed_cache_control, CC.FLAGS_EXPAND_PERPENDICULAR )
#
@ -1365,8 +1365,6 @@ class EditSubscriptionPanel( ClientGUIScrolledPanels.EditPanel ):
#
self._control_panel.AddF( self._seed_cache_button, CC.FLAGS_LONE_BUTTON )
rows = []
rows.append( ( 'currently paused: ', self._paused ) )
@ -1511,16 +1509,7 @@ class EditSubscriptionPanel( ClientGUIScrolledPanels.EditPanel ):
def _UpdateSeedInfo( self ):
seed_cache_text = HydrusData.ConvertIntToPrettyString( self._seed_cache.GetSeedCount() ) + ' urls in cache'
num_failed = self._seed_cache.GetSeedCount( CC.STATUS_FAILED )
if num_failed > 0:
seed_cache_text += ', ' + HydrusData.ConvertIntToPrettyString( num_failed ) + ' failed'
self._seed_info_st.SetLabelText( seed_cache_text )
self._seed_cache_control.SetSeedCache( self._seed_cache )
def _PresentForSiteType( self ):

154
include/ClientGUIScrolledPanelsManagement.py
View File

@ -2746,7 +2746,7 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
self._default_sort = ClientGUICommon.ChoiceSort( self )
self._sort_fallback = ClientGUICommon.ChoiceSort( self )
self._fallback_sort = ClientGUICommon.ChoiceSort( self )
self._default_collect = ClientGUICommon.CheckboxCollect( self )
@ -2758,22 +2758,28 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
#
self._new_options = HG.client_controller.GetNewOptions()
try:
self._default_sort.SetSelection( HC.options[ 'default_sort' ] )
self._default_sort.SetSort( self._new_options.GetDefaultSort() )
except:
self._default_sort.SetSelection( 0 )
media_sort = ClientMedia.MediaSort( ( 'system', CC.SORT_FILES_BY_FILESIZE ), CC.SORT_ASC )
self._default_sort.SetSort( media_sort )
try:
self._sort_fallback.SetSelection( HC.options[ 'sort_fallback' ] )
self._fallback_sort.SetSort( self._new_options.GetFallbackSort() )
except:
self._sort_fallback.SetSelection( 0 )
media_sort = ClientMedia.MediaSort( ( 'system', CC.SORT_FILES_BY_IMPORT_TIME ), CC.SORT_ASC )
self._fallback_sort.SetSort( media_sort )
for ( sort_by_type, sort_by ) in HC.options[ 'sort_by' ]:
@ -2786,7 +2792,7 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
rows = []
rows.append( ( 'Default sort: ', self._default_sort ) )
rows.append( ( 'Secondary sort (when primary gives two equal values): ', self._sort_fallback ) )
rows.append( ( 'Secondary sort (when primary gives two equal values): ', self._fallback_sort ) )
rows.append( ( 'Default collect: ', self._default_collect ) )
gridbox = ClientGUICommon.WrapInGrid( self, rows )
@ -2845,8 +2851,8 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
def UpdateOptions( self ):
HC.options[ 'default_sort' ] = self._default_sort.GetSelection()
HC.options[ 'sort_fallback' ] = self._sort_fallback.GetSelection()
self._new_options.SetDefaultSort( self._default_sort.GetSort() )
self._new_options.SetFallbackSort( self._fallback_sort.GetSort() )
HC.options[ 'default_collect' ] = self._default_collect.GetChoice()
sort_by_choices = []
@ -4641,9 +4647,9 @@ class ManageSubscriptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
#
columns = [ ( 'name', -1 ), ( 'site', 80 ), ( 'period', 80 ), ( 'last checked', 100 ), ( 'recent error?', 100 ), ( 'recent delay?', 100 ), ( 'urls', 60 ), ( 'failures', 60 ), ( 'paused', 80 ), ( 'check now?', 100 ) ]
columns = [ ( 'name', -1 ), ( 'site', 12 ), ( 'period', 9 ), ( 'last checked', 15 ), ( 'recent error?', 12 ), ( 'recent delay?', 12 ), ( 'urls', 8 ), ( 'failures', 8 ), ( 'paused', 8 ), ( 'check now?', 10 ) ]
self._subscriptions = ClientGUIListCtrl.SaneListCtrlForSingleObject( self, 300, columns, delete_key_callback = self.Delete, activation_callback = self.Edit )
self._subscriptions = ClientGUIListCtrl.BetterListCtrl( self, 'subscriptions', 25, 20, columns, self._ConvertSubscriptionToListCtrlTuples, delete_key_callback = self.Delete, activation_callback = self.Edit )
self._add = ClientGUICommon.BetterButton( self, 'add', self.Add )
@ -4675,9 +4681,7 @@ class ManageSubscriptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
for subscription in subscriptions:
( display_tuple, sort_tuple ) = self._ConvertSubscriptionToTuples( subscription )
self._subscriptions.Append( display_tuple, sort_tuple, subscription )
self._subscriptions.AddData( subscription )
#
@ -4714,7 +4718,7 @@ class ManageSubscriptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
self.SetSizer( vbox )
def _ConvertSubscriptionToTuples( self, subscription ):
def _ConvertSubscriptionToListCtrlTuples( self, subscription ):
( name, gallery_identifier, gallery_stream_identifiers, query, period, get_tags_if_url_known_and_file_redundant, initial_file_limit, periodic_file_limit, paused, import_file_options, import_tag_options, last_checked, last_error, check_now, seed_cache ) = subscription.ToTuple()
@ -4778,11 +4782,20 @@ class ManageSubscriptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
return ( display_tuple, sort_tuple )
def _GetExistingNames( self ):
subscriptions = self._subscriptions.GetData()
names = { subscription.GetName() for subscription in subscriptions }
return names
def _GetExportObject( self ):
to_export = HydrusSerialisable.SerialisableList()
for subscription in self._subscriptions.GetObjects( only_selected = True ):
for subscription in self._subscriptions.GetData( only_selected = True ):
to_export.append( subscription )
@ -4816,11 +4829,9 @@ class ManageSubscriptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
subscription = obj
self._subscriptions.SetNonDupeName( subscription )
subscription.SetNonDupeName( self._GetExistingNames() )
( display_tuple, sort_tuple ) = self._ConvertSubscriptionToTuples( subscription )
self._subscriptions.Append( display_tuple, sort_tuple, subscription )
self._subscriptions.AddData( subscription )
else:
@ -4843,36 +4854,32 @@ class ManageSubscriptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
new_subscription = panel.GetValue()
self._subscriptions.SetNonDupeName( new_subscription )
new_subscription.SetNonDupeName( self._GetExistingNames() )
( display_tuple, sort_tuple ) = self._ConvertSubscriptionToTuples( new_subscription )
self._subscriptions.Append( display_tuple, sort_tuple, new_subscription )
self._subscriptions.AddData( new_subscription )
def CheckNow( self ):
for i in self._subscriptions.GetAllSelected():
subscription = self._subscriptions.GetObject( i )
subscriptions = self._subscriptions.GetData( only_selected = True )
for subscription in subscriptions:
subscription.CheckNow()
( display_tuple, sort_tuple ) = self._ConvertSubscriptionToTuples( subscription )
self._subscriptions.UpdateRow( i, display_tuple, sort_tuple, subscription )
self._subscriptions.UpdateDatas( subscriptions )
def CommitChanges( self ):
subscriptions = self._subscriptions.GetObjects()
subscriptions = self._subscriptions.GetData()
HG.client_controller.Write( 'serialisables_overwrite', [ HydrusSerialisable.SERIALISABLE_TYPE_SUBSCRIPTION ], subscriptions )
# we pubsub changes outside, so it happens even on cancel
# we pubsub daemon wake outside, so not needed here--it happens even on cancel
def Delete( self ):
@ -4881,37 +4888,30 @@ class ManageSubscriptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
if dlg.ShowModal() == wx.ID_YES:
self._subscriptions.RemoveAllSelected()
self._subscriptions.DeleteSelected()
def Duplicate( self ):
subs_to_dupe = []
for subscription in self._subscriptions.GetObjects( only_selected = True ):
subs_to_dupe.append( subscription )
subs_to_dupe = self._subscriptions.GetData( only_selected = True )
for subscription in subs_to_dupe:
dupe_subscription = subscription.Duplicate()
self._subscriptions.SetNonDupeName( dupe_subscription )
dupe_subscription.SetNonDupeName( self._GetExistingNames() )
( display_tuple, sort_tuple ) = self._ConvertSubscriptionToTuples( dupe_subscription )
self._subscriptions.Append( display_tuple, sort_tuple, dupe_subscription )
self._subscriptions.AddData( dupe_subscription )
def Edit( self ):
for index in self._subscriptions.GetAllSelected():
subscription = self._subscriptions.GetObject( index )
subs_to_edit = self._subscriptions.GetData( only_selected = True )
for subscription in subs_to_edit:
with ClientGUITopLevelWindows.DialogEdit( self, 'edit subscription' ) as dlg:
@ -4925,25 +4925,23 @@ class ManageSubscriptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
if result == wx.ID_OK:
self._subscriptions.DeleteDatas( ( subscription, ) )
edited_subscription = panel.GetValue()
if edited_subscription.GetName() != original_name:
self._subscriptions.SetNonDupeName( edited_subscription )
edited_subscription.SetNonDupeName( self._GetExistingNames() )
( display_tuple, sort_tuple ) = self._ConvertSubscriptionToTuples( edited_subscription )
self._subscriptions.UpdateRow( index, display_tuple, sort_tuple, edited_subscription )
self._subscriptions.AddData( edited_subscription )
elif result == wx.ID_CANCEL:
break
self._subscriptions.Sort()
def ExportToClipboard( self ):
@ -5038,16 +5036,14 @@ class ManageSubscriptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
def PauseResume( self ):
for i in self._subscriptions.GetAllSelected():
subscription = self._subscriptions.GetObject( i )
subscriptions = self._subscriptions.GetData( only_selected = True )
for subscription in subscriptions:
subscription.PauseResume()
( display_tuple, sort_tuple ) = self._ConvertSubscriptionToTuples( subscription )
self._subscriptions.UpdateRow( i, display_tuple, sort_tuple, subscription )
self._subscriptions.UpdateDatas( subscriptions )
def Reset( self ):
@ -5058,25 +5054,23 @@ class ManageSubscriptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
if dlg.ShowModal() == wx.ID_YES:
for i in self._subscriptions.GetAllSelected():
subscription = self._subscriptions.GetObject( i )
subscriptions = self._subscriptions.GetData( only_selected = True )
for subscription in subscriptions:
subscription.Reset()
( display_tuple, sort_tuple ) = self._ConvertSubscriptionToTuples( subscription )
self._subscriptions.UpdateRow( i, display_tuple, sort_tuple, subscription )
self._subscriptions.UpdateDatas( subscriptions )
def RetryFailures( self ):
for i in self._subscriptions.GetAllSelected():
subscription = self._subscriptions.GetObject( i )
subscriptions = self._subscriptions.GetData( only_selected = True )
for subscription in subscriptions:
seed_cache = subscription.GetSeedCache()
@ -5087,24 +5081,20 @@ class ManageSubscriptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
seed_cache.UpdateSeedStatus( seed, CC.STATUS_UNKNOWN )
( display_tuple, sort_tuple ) = self._ConvertSubscriptionToTuples( subscription )
self._subscriptions.UpdateRow( i, display_tuple, sort_tuple, subscription )
self._subscriptions.UpdateDatas( subscriptions )
def ScrubDelays( self ):
for i in self._subscriptions.GetAllSelected():
subscription = self._subscriptions.GetObject( i )
subscriptions = self._subscriptions.GetData( only_selected = True )
for subscription in subscriptions:
subscription.ScrubDelay()
( display_tuple, sort_tuple ) = self._ConvertSubscriptionToTuples( subscription )
self._subscriptions.UpdateRow( i, display_tuple, sort_tuple, subscription )
self._subscriptions.UpdateDatas( subscriptions )
class ManageTagsPanel( ClientGUIScrolledPanels.ManagePanel ):

2
include/ClientGUIScrolledPanelsReview.py
View File

@ -1460,7 +1460,7 @@ class MigrateDatabasePanel( ClientGUIScrolledPanels.ReviewPanel ):
threading.Thread( target = THREADMigrateDatabase, args = ( self._controller, source, portable_locations, dest ) ).start()
HG.client_controller.CallToThreadLongRunning( THREADMigrateDatabase, self._controller, source, portable_locations, dest )

43
include/ClientGUISeedCache.py
View File

@ -24,10 +24,9 @@ class EditSeedCachePanel( ClientGUIScrolledPanels.EditPanel ):
# add index control row here, hide it if needed and hook into showing/hiding and postsizechangedevent on seed add/remove
height = 300
columns = [ ( 'source', -1 ), ( 'status', 90 ), ( 'added', 150 ), ( 'last modified', 150 ), ( 'note', 200 ) ]
columns = [ ( 'source', -1 ), ( 'status', 12 ), ( 'added', 20 ), ( 'last modified', 20 ), ( 'note', 30 ) ]
self._list_ctrl = ClientGUIListCtrl.SaneListCtrlForSingleObject( self, height, columns )
self._list_ctrl = ClientGUIListCtrl.BetterListCtrl( self, 'seed_cache', 30, 30, columns, self._ConvertSeedToListCtrlTuples )
#
@ -53,15 +52,11 @@ class EditSeedCachePanel( ClientGUIScrolledPanels.EditPanel ):
for seed in seeds:
sort_tuple = self._seed_cache.GetSeedInfo( seed )
( display_tuple, sort_tuple ) = self._GetListCtrlTuples( seed )
self._list_ctrl.Append( display_tuple, sort_tuple, seed )
self._list_ctrl.AddData( seed )
def _GetListCtrlTuples( self, seed ):
def _ConvertSeedToListCtrlTuples( self, seed ):
sort_tuple = self._seed_cache.GetSeedInfo( seed )
@ -82,7 +77,7 @@ class EditSeedCachePanel( ClientGUIScrolledPanels.EditPanel ):
notes = []
for seed in self._list_ctrl.GetObjects( only_selected = True ):
for seed in self._list_ctrl.GetData( only_selected = True ):
( seed, status, added_timestamp, last_modified_timestamp, note ) = self._seed_cache.GetSeedInfo( seed )
@ -104,7 +99,7 @@ class EditSeedCachePanel( ClientGUIScrolledPanels.EditPanel ):
def _CopySelectedSeeds( self ):
seeds = self._list_ctrl.GetObjects( only_selected = True )
seeds = self._list_ctrl.GetData( only_selected = True )
if len( seeds ) > 0:
@ -118,7 +113,7 @@ class EditSeedCachePanel( ClientGUIScrolledPanels.EditPanel ):
def _DeleteSelected( self ):
seeds_to_delete = self._list_ctrl.GetObjects( only_selected = True )
seeds_to_delete = self._list_ctrl.GetData( only_selected = True )
if len( seeds_to_delete ) > 0:
@ -136,14 +131,14 @@ class EditSeedCachePanel( ClientGUIScrolledPanels.EditPanel ):
def _SetSelected( self, status_to_set ):
seeds_to_set = self._list_ctrl.GetObjects( only_selected = True )
seeds_to_set = self._list_ctrl.GetData( only_selected = True )
self._seed_cache.UpdateSeedsStatus( seeds_to_set, status_to_set )
def _ShowMenuIfNeeded( self ):
if self._list_ctrl.GetSelectedItemCount() > 0:
if self._list_ctrl.HasSelected() > 0:
menu = wx.Menu()
@ -170,7 +165,7 @@ class EditSeedCachePanel( ClientGUIScrolledPanels.EditPanel ):
if self._seed_cache.HasSeed( seed ):
if self._list_ctrl.HasObject( seed ):
if self._list_ctrl.HasData( seed ):
seeds_to_update.append( seed )
@ -181,28 +176,16 @@ class EditSeedCachePanel( ClientGUIScrolledPanels.EditPanel ):
else:
if self._list_ctrl.HasObject( seed ):
if self._list_ctrl.HasData( seed ):
seeds_to_delete.append( seed )
for seed in seeds_to_delete:
index = self._list_ctrl.GetIndexFromObject( seed )
self._list_ctrl.DeleteItem( index )
self._list_ctrl.DeleteDatas( seeds_to_delete )
for seed in seeds_to_update:
index = self._list_ctrl.GetIndexFromObject( seed )
( display_tuple, sort_tuple ) = self._GetListCtrlTuples( seed )
self._list_ctrl.UpdateRow( index, display_tuple, sort_tuple, seed )
self._list_ctrl.UpdateDatas( seeds_to_update )
self._AddSeeds( seeds_to_add )

41
include/ClientImporting.py
View File

@ -1020,8 +1020,8 @@ class GalleryImport( HydrusSerialisable.SerialisableBase ):
def Start( self, page_key ):
threading.Thread( target = self._THREADWorkOnGallery, args = ( page_key, ) ).start()
threading.Thread( target = self._THREADWorkOnFiles, args = ( page_key, ) ).start()
HG.client_controller.CallToThreadLongRunning( self._THREADWorkOnGallery, page_key )
HG.client_controller.CallToThreadLongRunning( self._THREADWorkOnFiles, page_key )
HydrusSerialisable.SERIALISABLE_TYPES_TO_OBJECT_TYPES[ HydrusSerialisable.SERIALISABLE_TYPE_GALLERY_IMPORT ] = GalleryImport
@ -1271,7 +1271,7 @@ class HDDImport( HydrusSerialisable.SerialisableBase ):
def Start( self, page_key ):
threading.Thread( target = self._THREADWork, args = ( page_key, ) ).start()
HG.client_controller.CallToThreadLongRunning( self._THREADWork, page_key )
HydrusSerialisable.SERIALISABLE_TYPES_TO_OBJECT_TYPES[ HydrusSerialisable.SERIALISABLE_TYPE_HDD_IMPORT ] = HDDImport
@ -1309,9 +1309,9 @@ class ImportFolder( HydrusSerialisable.SerialisableBaseNamed ):
actions = {}
actions[ CC.STATUS_SUCCESSFUL ] = CC.IMPORT_FOLDER_DELETE
actions[ CC.STATUS_REDUNDANT ] = CC.IMPORT_FOLDER_DELETE
actions[ CC.STATUS_DELETED ] = CC.IMPORT_FOLDER_DELETE
actions[ CC.STATUS_SUCCESSFUL ] = CC.IMPORT_FOLDER_IGNORE
actions[ CC.STATUS_REDUNDANT ] = CC.IMPORT_FOLDER_IGNORE
actions[ CC.STATUS_DELETED ] = CC.IMPORT_FOLDER_IGNORE
actions[ CC.STATUS_FAILED ] = CC.IMPORT_FOLDER_IGNORE
@ -2300,8 +2300,8 @@ class PageOfImagesImport( HydrusSerialisable.SerialisableBase ):
def Start( self, page_key ):
threading.Thread( target = self._THREADWorkOnQueue, args = ( page_key, ) ).start()
threading.Thread( target = self._THREADWorkOnFiles, args = ( page_key, ) ).start()
HG.client_controller.CallToThreadLongRunning( self._THREADWorkOnQueue, page_key )
HG.client_controller.CallToThreadLongRunning( self._THREADWorkOnFiles, page_key )
HydrusSerialisable.SERIALISABLE_TYPES_TO_OBJECT_TYPES[ HydrusSerialisable.SERIALISABLE_TYPE_PAGE_OF_IMAGES_IMPORT ] = PageOfImagesImport
@ -3047,10 +3047,21 @@ class Subscription( HydrusSerialisable.SerialisableBaseNamed ):
example_nj = network_job_factory( 'GET', url )
p4 = not HG.client_controller.network_engine.bandwidth_manager.CanDoWork( example_nj.GetNetworkContexts() )
# just a little padding, to make sure we don't accidentally get into a long wait because we need to fetch file and tags independantly etc...
expected_requests = 3
expected_bytes = 1048576
p4 = not HG.client_controller.network_engine.bandwidth_manager.CanDoWork( example_nj.GetNetworkContexts(), expected_requests, expected_bytes )
if p1 or p3 or p4:
if p4:
job_key.SetVariable( 'popup_text_1', 'no more bandwidth to download files, so stopping for now' )
time.sleep( 2 )
break
@ -3212,7 +3223,11 @@ class Subscription( HydrusSerialisable.SerialisableBaseNamed ):
example_nj = network_job_factory( 'GET', url )
if HG.client_controller.network_engine.bandwidth_manager.CanDoWork( example_nj.GetNetworkContexts() ):
# just a little padding here
expected_requests = 3
expected_bytes = 1048576
if HG.client_controller.network_engine.bandwidth_manager.CanDoWork( example_nj.GetNetworkContexts(), expected_requests, expected_bytes ):
return True
@ -4151,8 +4166,8 @@ class ThreadWatcherImport( HydrusSerialisable.SerialisableBase ):
def Start( self, page_key ):
threading.Thread( target = self._THREADWorkOnThread, args = ( page_key, ) ).start()
threading.Thread( target = self._THREADWorkOnFiles, args = ( page_key, ) ).start()
HG.client_controller.CallToThreadLongRunning( self._THREADWorkOnThread, page_key )
HG.client_controller.CallToThreadLongRunning( self._THREADWorkOnFiles, page_key )
HydrusSerialisable.SERIALISABLE_TYPES_TO_OBJECT_TYPES[ HydrusSerialisable.SERIALISABLE_TYPE_THREAD_WATCHER_IMPORT ] = ThreadWatcherImport
@ -4453,7 +4468,7 @@ class URLsImport( HydrusSerialisable.SerialisableBase ):
def Start( self, page_key ):
threading.Thread( target = self._THREADWork, args = ( page_key, ) ).start()
HG.client_controller.CallToThreadLongRunning( self._THREADWork, page_key )
HydrusSerialisable.SERIALISABLE_TYPES_TO_OBJECT_TYPES[ HydrusSerialisable.SERIALISABLE_TYPE_URLS_IMPORT ] = URLsImport

617
include/ClientMedia.py
View File

@ -16,6 +16,7 @@ import HydrusData
import HydrusFileHandling
import HydrusExceptions
import HydrusGlobals as HG
import HydrusSerialisable
import itertools
def FlattenMedia( media_list ):
@ -220,7 +221,7 @@ def MergeTagsManagers( tags_managers ):
merged_service_keys_to_statuses_to_tags[ service_key ] = statuses_to_tags
return TagsManagerSimple( merged_service_keys_to_statuses_to_tags )
return TagsManager( merged_service_keys_to_statuses_to_tags )
class DuplicatesManager( object ):
@ -601,7 +602,7 @@ class MediaList( object ):
self._hashes = set()
self._sort_by = CC.SORT_BY_SMALLEST
self._media_sort = MediaSort( ( 'system', CC.SORT_FILES_BY_FILESIZE ), CC.SORT_ASC )
self._collect_by = []
self._collect_map_singletons = {}
@ -701,186 +702,17 @@ class MediaList( object ):
else: return self._sorted_media[ previous_index ]
def _GetSortFunction( self, sort_by ):
reverse = False
( sort_by_type, sort_by_data ) = sort_by
def deal_with_none( x ):
if x is None: return -1
else: return x
if sort_by_type == 'system':
if sort_by_data == CC.SORT_BY_RANDOM:
def sort_key( x ):
return random.random()
elif sort_by_data in ( CC.SORT_BY_SMALLEST, CC.SORT_BY_LARGEST ):
def sort_key( x ):
return deal_with_none( x.GetSize() )
if sort_by_data == CC.SORT_BY_LARGEST:
reverse = True
elif sort_by_data in ( CC.SORT_BY_SHORTEST, CC.SORT_BY_LONGEST ):
def sort_key( x ):
return deal_with_none( x.GetDuration() )
if sort_by_data == CC.SORT_BY_LONGEST:
reverse = True
elif sort_by_data in ( CC.SORT_BY_OLDEST, CC.SORT_BY_NEWEST ):
file_service = HG.client_controller.services_manager.GetService( self._file_service_key )
file_service_type = file_service.GetServiceType()
if file_service_type == HC.LOCAL_FILE_DOMAIN:
file_service_key = CC.COMBINED_LOCAL_FILE_SERVICE_KEY
else:
file_service_key = self._file_service_key
def sort_key( x ):
return deal_with_none( x.GetTimestamp( file_service_key ) )
if sort_by_data == CC.SORT_BY_NEWEST:
reverse = True
elif sort_by_data in ( CC.SORT_BY_HEIGHT_ASC, CC.SORT_BY_HEIGHT_DESC ):
def sort_key( x ):
return deal_with_none( x.GetResolution()[1] )
if sort_by_data == CC.SORT_BY_HEIGHT_DESC:
reverse = True
elif sort_by_data in ( CC.SORT_BY_WIDTH_ASC, CC.SORT_BY_WIDTH_DESC ):
def sort_key( x ):
return deal_with_none( x.GetResolution()[0] )
if sort_by_data == CC.SORT_BY_WIDTH_DESC:
reverse = True
elif sort_by_data in ( CC.SORT_BY_RATIO_ASC, CC.SORT_BY_RATIO_DESC ):
def sort_key( x ):
( width, height ) = x.GetResolution()
if width is None or height is None or width == 0 or height == 0:
return -1
else:
return float( width ) / float( height )
if sort_by_data == CC.SORT_BY_RATIO_DESC:
reverse = True
elif sort_by_data in ( CC.SORT_BY_NUM_PIXELS_ASC, CC.SORT_BY_NUM_PIXELS_DESC ):
def sort_key( x ):
( width, height ) = x.GetResolution()
if width is None or height is None:
return -1
else:
return width * height
if sort_by_data == CC.SORT_BY_NUM_PIXELS_DESC:
reverse = True
elif sort_by_data == CC.SORT_BY_MIME:
def sort_key( x ):
return x.GetMime()
elif sort_by_type == 'namespaces':
namespaces = sort_by_data
def sort_key( x ):
x_tags_manager = x.GetTagsManager()
return [ x_tags_manager.GetComparableNamespaceSlice( ( namespace, ) ) for namespace in namespaces ]
elif sort_by_type in ( 'rating_descend', 'rating_ascend' ):
service_key = sort_by_data
def sort_key( x ):
x_ratings_manager = x.GetRatingsManager()
rating = deal_with_none( x_ratings_manager.GetRating( service_key ) )
return rating
if sort_by_type == 'rating_descend':
reverse = True
return ( sort_key, reverse )
def _HasHashes( self, hashes ):
return True in ( not hashes.isdisjoint( media.GetHashes() ) for media in self._sorted_media )
for hash in hashes:
if hash in self._hashes:
return True
return False
def _RecalcHashes( self ):
@ -990,7 +822,7 @@ class MediaList( object ):
collected_media = self._GenerateMediaCollection( [ media.GetMediaResult() for media in medias ] )
collected_media.Sort( self._sort_by )
collected_media.Sort( self._media_sort )
self._collected_media.add( collected_media )
self._collect_map_collected[ key ] = collected_media
@ -1005,7 +837,7 @@ class MediaList( object ):
collected_media.AddMedia( medias )
collected_media.Sort( self._sort_by )
collected_media.Sort( self._media_sort )
new_media.append( collected_media )
@ -1020,7 +852,7 @@ class MediaList( object ):
collected_media = self._GenerateMediaCollection( [ media.GetMediaResult() for media in medias ] )
collected_media.Sort( self._sort_by )
collected_media.Sort( self._media_sort )
self._collected_media.add( collected_media )
self._collect_map_collected[ key ] = collected_media
@ -1325,38 +1157,29 @@ class MediaList( object ):
def Sort( self, sort_by = None ):
def Sort( self, media_sort = None ):
for media in self._collected_media:
media.Sort( sort_by )
media.Sort( media_sort )
if sort_by is None:
if media_sort is None:
sort_by = self._sort_by
media_sort = self._media_sort
self._sort_by = sort_by
self._media_sort = media_sort
sort_choices = ClientData.GetSortChoices( add_namespaces_and_ratings = True )
media_sort_fallback = HG.client_controller.GetNewOptions().GetFallbackSort()
try:
sort_by_fallback = sort_choices[ HC.options[ 'sort_fallback' ] ]
except IndexError:
sort_by_fallback = sort_choices[ 0 ]
( sort_key, reverse ) = self._GetSortFunction( sort_by_fallback )
( sort_key, reverse ) = media_sort_fallback.GetSortKeyAndReverse( self._file_service_key )
self._sorted_media.sort( sort_key, reverse = reverse )
# this is a stable sort, so the fallback order above will remain for equal items
( sort_key, reverse ) = self._GetSortFunction( self._sort_by )
( sort_key, reverse ) = self._media_sort.GetSortKeyAndReverse( self._file_service_key )
self._sorted_media.sort( sort_key = sort_key, reverse = reverse )
@ -2046,6 +1869,300 @@ class MediaResult( object ):
return ( self._file_info_manager, self._tags_manager, self._locations_manager, self._ratings_manager )
class MediaSort( HydrusSerialisable.SerialisableBase ):
SERIALISABLE_TYPE = HydrusSerialisable.SERIALISABLE_TYPE_MEDIA_SORT
SERIALISABLE_VERSION = 1
def __init__( self, sort_type = None, sort_asc = None ):
if sort_type is None:
sort_type = ( 'system', CC.SORT_FILES_BY_FILESIZE )
if sort_asc is None:
sort_asc = CC.SORT_ASC
self.sort_type = sort_type
self.sort_asc = sort_asc
def _GetSerialisableInfo( self ):
( sort_metatype, sort_data ) = self.sort_type
if sort_metatype == 'system':
serialisable_sort_data = sort_data
elif sort_metatype == 'namespaces':
serialisable_sort_data = sort_data
elif sort_metatype == 'rating':
service_key = sort_data
serialisable_sort_data = service_key.encode( 'hex' )
return ( sort_metatype, serialisable_sort_data, self.sort_asc )
def _InitialiseFromSerialisableInfo( self, serialisable_info ):
( sort_metatype, serialisable_sort_data, self.sort_asc ) = serialisable_info
if sort_metatype == 'system':
sort_data = serialisable_sort_data
elif sort_metatype == 'namespaces':
sort_data = tuple( serialisable_sort_data )
elif sort_metatype == 'rating':
sort_data = serialisable_sort_data.decode( 'hex' )
self.sort_type = ( sort_metatype, sort_data )
def CanAsc( self ):
( sort_metatype, sort_data ) = self.sort_type
if sort_metatype == 'system':
if sort_data in ( CC.SORT_FILES_BY_MIME, CC.SORT_FILES_BY_RANDOM ):
return False
elif sort_metatype == 'namespaces':
return False
return True
def GetSortKeyAndReverse( self, file_service_key ):
reverse = False
( sort_metadata, sort_data ) = self.sort_type
def deal_with_none( x ):
if x is None: return -1
else: return x
if sort_metadata == 'system':
if sort_data == CC.SORT_FILES_BY_RANDOM:
def sort_key( x ):
return random.random()
elif sort_data == CC.SORT_FILES_BY_FILESIZE:
def sort_key( x ):
return deal_with_none( x.GetSize() )
elif sort_data == CC.SORT_FILES_BY_DURATION:
def sort_key( x ):
return deal_with_none( x.GetDuration() )
elif sort_data == CC.SORT_FILES_BY_IMPORT_TIME:
file_service = HG.client_controller.services_manager.GetService( file_service_key )
file_service_type = file_service.GetServiceType()
if file_service_type == HC.LOCAL_FILE_DOMAIN:
file_service_key = CC.COMBINED_LOCAL_FILE_SERVICE_KEY
def sort_key( x ):
return deal_with_none( x.GetTimestamp( file_service_key ) )
elif sort_data == CC.SORT_FILES_BY_HEIGHT:
def sort_key( x ):
return deal_with_none( x.GetResolution()[1] )
elif sort_data == CC.SORT_FILES_BY_WIDTH:
def sort_key( x ):
return deal_with_none( x.GetResolution()[0] )
elif sort_data == CC.SORT_FILES_BY_RATIO:
def sort_key( x ):
( width, height ) = x.GetResolution()
if width is None or height is None or width == 0 or height == 0:
return -1
else:
return float( width ) / float( height )
elif sort_data == CC.SORT_FILES_BY_NUM_PIXELS:
def sort_key( x ):
( width, height ) = x.GetResolution()
if width is None or height is None:
return -1
else:
return width * height
elif sort_data == CC.SORT_FILES_BY_NUM_TAGS:
def sort_key( x ):
tags_manager = x.GetTagsManager()
return( len( tags_manager.GetCurrent() ) + len( tags_manager.GetPending() ) )
elif sort_data == CC.SORT_FILES_BY_MIME:
def sort_key( x ):
return x.GetMime()
elif sort_metadata == 'namespaces':
namespaces = sort_data
def sort_key( x ):
x_tags_manager = x.GetTagsManager()
return [ x_tags_manager.GetComparableNamespaceSlice( ( namespace, ) ) for namespace in namespaces ]
elif sort_metadata == 'rating':
service_key = sort_data
def sort_key( x ):
x_ratings_manager = x.GetRatingsManager()
rating = deal_with_none( x_ratings_manager.GetRating( service_key ) )
return rating
return ( sort_key, self.sort_asc )
def GetSortTypeString( self ):
( sort_metatype, sort_data ) = self.sort_type
sort_string = 'sort by '
if sort_metatype == 'system':
sort_string_lookup = {}
sort_string_lookup[ CC.SORT_FILES_BY_FILESIZE ] = 'filesize'
sort_string_lookup[ CC.SORT_FILES_BY_DURATION ] = 'duration'
sort_string_lookup[ CC.SORT_FILES_BY_IMPORT_TIME ] = 'age'
sort_string_lookup[ CC.SORT_FILES_BY_MIME ] = 'mime'
sort_string_lookup[ CC.SORT_FILES_BY_RANDOM ] = 'random'
sort_string_lookup[ CC.SORT_FILES_BY_WIDTH ] = 'width'
sort_string_lookup[ CC.SORT_FILES_BY_HEIGHT ] = 'height'
sort_string_lookup[ CC.SORT_FILES_BY_RATIO ] = 'resolution ratio'
sort_string_lookup[ CC.SORT_FILES_BY_NUM_PIXELS ] = 'number of pixels'
sort_string_lookup[ CC.SORT_FILES_BY_NUM_TAGS ] = 'number of tags'
sort_string += sort_string_lookup[ sort_data ]
elif sort_metatype == 'namespaces':
namespaces = sort_data
sort_string += '-'.join( namespaces )
elif sort_metatype == 'rating':
service_key = sort_data
service = HG.client_controller.services_manager.GetService( service_key )
sort_string += service.GetName()
return sort_string
def GetSortAscStrings( self ):
( sort_metatype, sort_data ) = self.sort_type
if sort_metatype == 'system':
sort_string_lookup = {}
sort_string_lookup[ CC.SORT_FILES_BY_FILESIZE ] = ( 'smallest first', 'largest first' )
sort_string_lookup[ CC.SORT_FILES_BY_DURATION ] = ( 'shortest first', 'longest first' )
sort_string_lookup[ CC.SORT_FILES_BY_IMPORT_TIME ] = ( 'oldest first', 'newest first' )
sort_string_lookup[ CC.SORT_FILES_BY_MIME ] = ( 'mime', 'mime' )
sort_string_lookup[ CC.SORT_FILES_BY_RANDOM ] = ( 'random', 'random' )
sort_string_lookup[ CC.SORT_FILES_BY_WIDTH ] = ( 'slimmest first', 'widest first' )
sort_string_lookup[ CC.SORT_FILES_BY_HEIGHT ] = ( 'shortest first', 'tallest first' )
sort_string_lookup[ CC.SORT_FILES_BY_RATIO ] = ( 'tallest first', 'widest first' )
sort_string_lookup[ CC.SORT_FILES_BY_NUM_PIXELS ] = ( 'ascending', 'descending' )
sort_string_lookup[ CC.SORT_FILES_BY_NUM_TAGS ] = ( 'ascending', 'descending' )
return sort_string_lookup[ sort_data ]
else:
return ( 'ascending', 'descending' )
HydrusSerialisable.SERIALISABLE_TYPES_TO_OBJECT_TYPES[ HydrusSerialisable.SERIALISABLE_TYPE_MEDIA_SORT ] = MediaSort
class SortedList( object ):
def __init__( self, initial_items = None ):
@ -2075,12 +2192,12 @@ class SortedList( object ):
def __iter__( self ):
for item in self._sorted_list: yield item
return iter( self._sorted_list )
def __len__( self ):
return self._sorted_list.__len__()
return len( self._sorted_list )
def _DirtyIndices( self ):
@ -2249,6 +2366,30 @@ class TagsManagerSimple( object ):
return tuple( slice )
def GetCurrent( self, service_key = CC.COMBINED_TAG_SERVICE_KEY ):
if service_key == CC.COMBINED_TAG_SERVICE_KEY:
self._RecalcCombinedIfNeeded()
statuses_to_tags = self._service_keys_to_statuses_to_tags[ service_key ]
return set( statuses_to_tags[ HC.CONTENT_STATUS_CURRENT ] )
def GetDeleted( self, service_key = CC.COMBINED_TAG_SERVICE_KEY ):
if service_key == CC.COMBINED_TAG_SERVICE_KEY:
self._RecalcCombinedIfNeeded()
statuses_to_tags = self._service_keys_to_statuses_to_tags[ service_key ]
return set( statuses_to_tags[ HC.CONTENT_STATUS_DELETED ] )
def GetNamespaceSlice( self, namespaces ):
self._RecalcCombinedIfNeeded()
@ -2267,6 +2408,30 @@ class TagsManagerSimple( object ):
return slice
def GetPending( self, service_key = CC.COMBINED_TAG_SERVICE_KEY ):
if service_key == CC.COMBINED_TAG_SERVICE_KEY:
self._RecalcCombinedIfNeeded()
statuses_to_tags = self._service_keys_to_statuses_to_tags[ service_key ]
return set( statuses_to_tags[ HC.CONTENT_STATUS_PENDING ] )
def GetPetitioned( self, service_key = CC.COMBINED_TAG_SERVICE_KEY ):
if service_key == CC.COMBINED_TAG_SERVICE_KEY:
self._RecalcCombinedIfNeeded()
statuses_to_tags = self._service_keys_to_statuses_to_tags[ service_key ]
return set( statuses_to_tags[ HC.CONTENT_STATUS_PETITIONED ] )
class TagsManager( TagsManagerSimple ):
def __init__( self, service_keys_to_statuses_to_tags ):
@ -2343,30 +2508,6 @@ class TagsManager( TagsManagerSimple ):
return TagsManager( dupe_service_keys_to_statuses_to_tags )
def GetCurrent( self, service_key = CC.COMBINED_TAG_SERVICE_KEY ):
if service_key == CC.COMBINED_TAG_SERVICE_KEY:
self._RecalcCombinedIfNeeded()
statuses_to_tags = self._service_keys_to_statuses_to_tags[ service_key ]
return set( statuses_to_tags[ HC.CONTENT_STATUS_CURRENT ] )
def GetDeleted( self, service_key = CC.COMBINED_TAG_SERVICE_KEY ):
if service_key == CC.COMBINED_TAG_SERVICE_KEY:
self._RecalcCombinedIfNeeded()
statuses_to_tags = self._service_keys_to_statuses_to_tags[ service_key ]
return set( statuses_to_tags[ HC.CONTENT_STATUS_DELETED ] )
def GetNumTags( self, service_key, include_current_tags = True, include_pending_tags = False ):
if service_key == CC.COMBINED_TAG_SERVICE_KEY:
@ -2384,30 +2525,6 @@ class TagsManager( TagsManagerSimple ):
return num_tags
def GetPending( self, service_key = CC.COMBINED_TAG_SERVICE_KEY ):
if service_key == CC.COMBINED_TAG_SERVICE_KEY:
self._RecalcCombinedIfNeeded()
statuses_to_tags = self._service_keys_to_statuses_to_tags[ service_key ]
return set( statuses_to_tags[ HC.CONTENT_STATUS_PENDING ] )
def GetPetitioned( self, service_key = CC.COMBINED_TAG_SERVICE_KEY ):
if service_key == CC.COMBINED_TAG_SERVICE_KEY:
self._RecalcCombinedIfNeeded()
statuses_to_tags = self._service_keys_to_statuses_to_tags[ service_key ]
return set( statuses_to_tags[ HC.CONTENT_STATUS_PETITIONED ] )
def GetServiceKeysToStatusesToTags( self ):
self._RecalcCombinedIfNeeded()

79
include/ClientNetworking.py
View File

@ -390,7 +390,7 @@ class HTTPConnectionManager( object ):
self._lock = threading.Lock()
threading.Thread( target = self.DAEMONMaintainConnections, name = 'Maintain Connections' ).start()
HG.client_controller.CallToThreadLongRunning( self.DAEMONMaintainConnections )
def _DoRequest( self, method, location, path, query, request_headers, body, follow_redirects = True, report_hooks = None, temp_path = None, hydrus_network = False, num_redirects_permitted = 4 ):
@ -1177,7 +1177,7 @@ class NetworkBandwidthManager( HydrusSerialisable.SerialisableBase ):
def CanDoWork( self, network_contexts ):
def CanDoWork( self, network_contexts, expected_requests, expected_bytes ):
with self._lock:
@ -1187,7 +1187,7 @@ class NetworkBandwidthManager( HydrusSerialisable.SerialisableBase ):
bandwidth_tracker = self._network_contexts_to_bandwidth_trackers[ network_context ]
if not bandwidth_rules.CanDoWork( bandwidth_tracker ):
if not bandwidth_rules.CanDoWork( bandwidth_tracker, expected_requests, expected_bytes ):
return False
@ -1277,22 +1277,6 @@ class NetworkBandwidthManager( HydrusSerialisable.SerialisableBase ):
def GetEstimateInfo( self, network_contexts ):
with self._lock:
# something that returns ( 'about a minute until you can request again', 60 )
# figure out the longest estimate from the rules and trackers
# make some pretty text out of that
# return ( text, seconds )
pass
def GetNetworkContextsForUser( self, history_time_delta_threshold = None ):
with self._lock:
@ -1344,6 +1328,32 @@ class NetworkBandwidthManager( HydrusSerialisable.SerialisableBase ):
def GetWaitingEstimate( self, network_contexts ):
with self._lock:
estimates = []
for network_context in network_contexts:
bandwidth_rules = self._GetRules( network_context )
bandwidth_tracker = self._network_contexts_to_bandwidth_trackers[ network_context ]
estimates.append( bandwidth_rules.GetWaitingEstimate( bandwidth_tracker ) )
if len( estimates ) == 0:
return 0
else:
return max( estimates )
def IsDirty( self ):
with self._lock:
@ -1620,8 +1630,6 @@ class NetworkEngine( object ):
elif not job.BandwidthOK():
job.SetStatus( u'waiting on bandwidth\u2026' )
return True
else:
@ -2026,10 +2034,33 @@ class NetworkJob( object ):
if not result:
self._status_text = u'waiting on bandwidth\u2026' # add the 'waiting ~4 minutes' text stuff here
waiting_duration = self.engine.bandwidth_manager.GetWaitingEstimate( self._network_contexts )
# if the time to wait > 10s:
# self._Sleep( 10 )
if waiting_duration <= 1:
self._status_text = ''
else:
pending_timestamp = HydrusData.GetNow() + waiting_duration
waiting_str = HydrusData.ConvertTimestampToPrettyPending( pending_timestamp )
self._status_text = u'bandwidth free ' + waiting_str + u'\u2026'
if waiting_duration > 1200:
self._Sleep( 30 )
elif waiting_duration > 120:
self._Sleep( 10 )
elif waiting_duration > 10:
self._Sleep( 1 )
return result

2
include/HydrusConstants.py
View File

@ -49,7 +49,7 @@ options = {}
# Misc
NETWORK_VERSION = 18
SOFTWARE_VERSION = 267
SOFTWARE_VERSION = 268
UNSCALED_THUMBNAIL_DIMENSIONS = ( 200, 200 )

55
include/HydrusController.py
View File

@ -48,6 +48,7 @@ class HydrusController( object ):
self._managers = {}
self._call_to_threads = []
self._long_running_call_to_threads = []
self._timestamps = collections.defaultdict( lambda: 0 )
@ -56,7 +57,7 @@ class HydrusController( object ):
self._just_woke_from_sleep = False
self._system_busy = False
threading.Thread( target = self.DAEMONPubSub, name = 'Pubsub Daemon' ).start()
self.CallToThreadLongRunning( self.DAEMONPubSub )
def _GetCallToThread( self ):
@ -89,6 +90,25 @@ class HydrusController( object ):
return call_to_thread
def _GetCallToThreadLongRunning( self ):
for call_to_thread in self._long_running_call_to_threads:
if not call_to_thread.CurrentlyWorking():
return call_to_thread
call_to_thread = HydrusThreading.THREADCallToThread( self )
self._long_running_call_to_threads.append( call_to_thread )
call_to_thread.start()
return call_to_thread
def _InitDB( self ):
raise NotImplementedError()
@ -125,7 +145,14 @@ class HydrusController( object ):
def pub( self, topic, *args, **kwargs ):
self._pubsub.pub( topic, *args, **kwargs )
if self._model_shutdown:
self._pubsub.pubimmediate( topic, *args, **kwargs )
else:
self._pubsub.pub( topic, *args, **kwargs )
def pubimmediate( self, topic, *args, **kwargs ):
@ -162,6 +189,30 @@ class HydrusController( object ):
call_to_thread.put( callable, *args, **kwargs )
def CallToThreadLongRunning( self, callable, *args, **kwargs ):
if HG.callto_report_mode:
what_to_report = [ callable ]
if len( args ) > 0:
what_to_report.append( args )
if len( kwargs ) > 0:
what_to_report.append( kwargs )
HydrusData.ShowText( tuple( what_to_report ) )
call_to_thread = self._GetCallToThreadLongRunning()
call_to_thread.put( callable, *args, **kwargs )
def ClearCaches( self ):
for cache in self._caches.values(): cache.Clear()

2
include/HydrusDB.py
View File

@ -227,7 +227,7 @@ class HydrusDB( object ):
self._CloseDBCursor()
threading.Thread( target = self.MainLoop, name = 'Database Main Loop' ).start()
self._controller.CallToThreadLongRunning( self.MainLoop )
while not self._ready_to_serve_requests:

3
include/HydrusData.py
View File

@ -501,7 +501,8 @@ def ConvertTimestampToPrettyPending( timestamp ):
elif months > 0: return 'in ' + ' '.join( ( mo, d ) )
elif days > 0: return 'in ' + ' '.join( ( d, h ) )
elif hours > 0: return 'in ' + ' '.join( ( h, m ) )
else: return 'in ' + ' '.join( ( m, s ) )
elif minutes > 0: return 'in ' + ' '.join( ( m, s ) )
else: return 'in ' + s
def ConvertTimestampToPrettySync( timestamp ):

157
include/HydrusNetworking.py
View File

@ -124,7 +124,7 @@ class BandwidthRules( HydrusSerialisable.SerialisableBase ):
continue
if bandwidth_tracker.GetUsage( bandwidth_type, time_delta ) > max_allowed:
if bandwidth_tracker.GetUsage( bandwidth_type, time_delta ) >= max_allowed:
return False
@ -134,7 +134,7 @@ class BandwidthRules( HydrusSerialisable.SerialisableBase ):
def CanDoWork( self, bandwidth_tracker, threshold = 30 ):
def CanDoWork( self, bandwidth_tracker, expected_requests, expected_bytes, threshold = 30 ):
with self._lock:
@ -146,6 +146,16 @@ class BandwidthRules( HydrusSerialisable.SerialisableBase ):
continue
# we don't want to do a tiny amount of work, we want to do a decent whack
if bandwidth_type == HC.BANDWIDTH_TYPE_REQUESTS:
max_allowed -= expected_requests
elif bandwidth_type == HC.BANDWIDTH_TYPE_DATA:
max_allowed -= expected_bytes
if bandwidth_tracker.GetUsage( bandwidth_type, time_delta ) >= max_allowed:
return False
@ -180,6 +190,31 @@ class BandwidthRules( HydrusSerialisable.SerialisableBase ):
def GetWaitingEstimate( self, bandwidth_tracker ):
with self._lock:
estimates = []
for ( bandwidth_type, time_delta, max_allowed ) in self._rules:
if bandwidth_tracker.GetUsage( bandwidth_type, time_delta ) >= max_allowed:
estimates.append( bandwidth_tracker.GetWaitingEstimate( bandwidth_type, time_delta, max_allowed ) )
if len( estimates ) == 0:
return 0
else:
return max( estimates )
def GetUsageStringsAndGaugeTuples( self, bandwidth_tracker, threshold = 600 ):
with self._lock:
@ -311,34 +346,7 @@ class BandwidthTracker( HydrusSerialisable.SerialisableBase ):
return datetime.datetime.utcfromtimestamp( HydrusData.GetNow() )
def _GetMonthTime( self, dt ):
( year, month ) = ( dt.year, dt.month )
month_dt = datetime.datetime( year, month, 1 )
month_time = calendar.timegm( month_dt.timetuple() )
return month_time
def _GetRawUsage( self, bandwidth_type, time_delta ):
if time_delta is None:
dt = self._GetCurrentDateTime()
month_time = self._GetMonthTime( dt )
if bandwidth_type == HC.BANDWIDTH_TYPE_DATA:
return self._months_bytes[ month_time ]
elif bandwidth_type == HC.BANDWIDTH_TYPE_REQUESTS:
return self._months_requests[ month_time ]
def _GetWindowAndCounter( self, bandwidth_type, time_delta ):
if bandwidth_type == HC.BANDWIDTH_TYPE_DATA:
@ -387,6 +395,40 @@ class BandwidthTracker( HydrusSerialisable.SerialisableBase ):
return ( window, counter )
def _GetMonthTime( self, dt ):
( year, month ) = ( dt.year, dt.month )
month_dt = datetime.datetime( year, month, 1 )
month_time = calendar.timegm( month_dt.timetuple() )
return month_time
def _GetRawUsage( self, bandwidth_type, time_delta ):
if time_delta is None:
dt = self._GetCurrentDateTime()
month_time = self._GetMonthTime( dt )
if bandwidth_type == HC.BANDWIDTH_TYPE_DATA:
return self._months_bytes[ month_time ]
elif bandwidth_type == HC.BANDWIDTH_TYPE_REQUESTS:
return self._months_requests[ month_time ]
( window, counter ) = self._GetWindowAndCounter( bandwidth_type, time_delta )
# we need the 'window' because this tracks brackets from the first timestamp and we want to include if 'since' lands anywhere in the bracket
# e.g. if it is 1200 and we want the past 1,000, we also need the bracket starting at 0, which will include 200-999
@ -549,6 +591,59 @@ class BandwidthTracker( HydrusSerialisable.SerialisableBase ):
def GetWaitingEstimate( self, bandwidth_type, time_delta, max_allowed ):
with self._lock:
if time_delta is None: # this is monthly
dt = self._GetCurrentDateTime()
( year, month ) = ( dt.year, dt.month )
next_month_dt = datetime.datetime( year, month + 1, 1 )
next_month_time = calendar.timegm( next_month_dt.timetuple() )
return next_month_time - HydrusData.GetNow()
else:
# we want the highest time_delta at which usage is >= than max_allowed
# time_delta subtract that amount is the time we have to wait for usage to be less than max_allowed
# e.g. if in the past 24 hours there was a bunch of usage 16 hours ago clogging it up, we'll have to wait ~8 hours
( window, counter ) = self._GetWindowAndCounter( bandwidth_type, time_delta )
time_and_values = counter.items()
time_and_values.sort( reverse = True )
now = HydrusData.GetNow()
usage = 0
for ( timestamp, value ) in time_and_values:
current_search_time_delta = now - timestamp
if current_search_time_delta > time_delta: # we are searching beyond our time delta. no need to wait
break
usage += value
if usage >= max_allowed:
return time_delta - current_search_time_delta
return 0
def ReportDataUsed( self, num_bytes ):
with self._lock:
@ -592,5 +687,5 @@ class BandwidthTracker( HydrusSerialisable.SerialisableBase ):
self._MaintainCache()
HydrusSerialisable.SERIALISABLE_TYPES_TO_OBJECT_TYPES[ HydrusSerialisable.SERIALISABLE_TYPE_BANDWIDTH_TRACKER ] = BandwidthTracker

18
include/HydrusSerialisable.py
View File

@ -51,6 +51,7 @@ SERIALISABLE_TYPE_NETWORK_BANDWIDTH_MANAGER = 45
SERIALISABLE_TYPE_NETWORK_SESSION_MANAGER = 46
SERIALISABLE_TYPE_NETWORK_CONTEXT = 47
SERIALISABLE_TYPE_NETWORK_LOGIN_MANAGER = 48
SERIALISABLE_TYPE_MEDIA_SORT = 49
SERIALISABLE_TYPES_TO_OBJECT_TYPES = {}
@ -166,6 +167,23 @@ class SerialisableBaseNamed( SerialisableBase ):
def SetName( self, name ): self._name = name
def SetNonDupeName( self, disallowed_names ):
i = 1
new_name = self._name
original_name = self._name
while new_name in disallowed_names:
new_name = original_name + ' (' + str( i ) + ')'
i += 1
self._name = new_name
class SerialisableDictionary( SerialisableBase, dict ):
SERIALISABLE_TYPE = SERIALISABLE_TYPE_DICTIONARY

92
include/HydrusThreading.py
View File

@ -75,66 +75,14 @@ class DAEMON( threading.Thread ):
self._event.set()
class DAEMONQueue( DAEMON ):
def __init__( self, controller, name, callable, queue_topic, period = 10 ):
DAEMON.__init__( self, controller, name )
self._callable = callable
self._queue = Queue.Queue()
self._queue_topic = queue_topic
self._period = period
self._controller.sub( self, 'put', queue_topic )
self.start()
def put( self, data ): self._queue.put( data )
def run( self ):
time.sleep( 3 )
while True:
while self._queue.empty():
if IsThreadShuttingDown():
return
self._event.wait( self._period )
self._event.clear()
items = []
while not self._queue.empty(): items.append( self._queue.get() )
try:
self._callable( self._controller, items )
except HydrusExceptions.ShutdownException:
return
except Exception as e:
HydrusData.ShowException( e )
class DAEMONWorker( DAEMON ):
def __init__( self, controller, name, callable, topics = None, period = 3600, init_wait = 3, pre_call_wait = 0 ):
if topics is None: topics = []
if topics is None:
topics = []
DAEMON.__init__( self, controller, name )
@ -144,7 +92,10 @@ class DAEMONWorker( DAEMON ):
self._init_wait = init_wait
self._pre_call_wait = pre_call_wait
for topic in topics: self._controller.sub( self, 'set', topic )
for topic in topics:
self._controller.sub( self, 'set', topic )
self.start()
@ -239,7 +190,7 @@ class THREADCallToThread( DAEMON ):
self._queue = Queue.Queue()
self._currently_working = False
self._currently_working = True # start off true so new threads aren't used twice by two quick successive calls
def CurrentlyWorking( self ):
@ -249,6 +200,8 @@ class THREADCallToThread( DAEMON ):
def put( self, callable, *args, **kwargs ):
self._currently_working = True
self._queue.put( ( callable, args, kwargs ) )
self._event.set()
@ -258,20 +211,21 @@ class THREADCallToThread( DAEMON ):
while True:
while self._queue.empty():
if self._controller.ModelIsShutdown(): return
self._event.wait( 1200 )
self._event.clear()
try:
( callable, args, kwargs ) = self._queue.get()
while self._queue.empty():
if self._controller.ModelIsShutdown():
return
self._event.wait( 1200 )
self._event.clear()
self._currently_working = True
( callable, args, kwargs ) = self._queue.get()
callable( *args, **kwargs )

9
include/TestClientDaemons.py
View File

@ -40,7 +40,14 @@ class TestDaemons( unittest.TestCase ):
#
import_folder = ClientImporting.ImportFolder( 'imp', path = test_dir )
actions = {}
actions[ CC.STATUS_SUCCESSFUL ] = CC.IMPORT_FOLDER_DELETE
actions[ CC.STATUS_REDUNDANT ] = CC.IMPORT_FOLDER_DELETE
actions[ CC.STATUS_DELETED ] = CC.IMPORT_FOLDER_DELETE
actions[ CC.STATUS_FAILED ] = CC.IMPORT_FOLDER_IGNORE
import_folder = ClientImporting.ImportFolder( 'imp', path = test_dir, actions = actions )
HG.test_controller.SetRead( 'serialisable_named', [ import_folder ] )

2
include/TestDB.py
View File

@ -690,7 +690,7 @@ class TestClientDB( unittest.TestCase ):
page_names.append( management_controller.GetPageName() )
print( page_names )
self.assertEqual( page_names, [ u'hentai foundry artist', u'import', u'thread watcher', u'page download', u'local tags petitions', u'search', u'search', u'files', u'wew lad', u'files' ] )

13
test.py
View File

@ -71,6 +71,10 @@ class Controller( object ):
HG.server_controller = self
HG.test_controller = self
self.gui = self
self._call_to_threads = []
self._pubsub = HydrusPubSub.HydrusPubSub( self )
self._new_options = ClientData.ClientOptions( self.db_dir )
@ -81,8 +85,6 @@ class Controller( object ):
self._http = ClientNetworking.HTTPConnectionManager()
self._call_to_threads = []
self._reads = {}
self._reads[ 'hydrus_sessions' ] = []
@ -193,6 +195,8 @@ class Controller( object ):
call_to_thread.put( callable, *args, **kwargs )
CallToThreadLongRunning = CallToThread
def DoHTTP( self, *args, **kwargs ): return self._http.Request( *args, **kwargs )
def GetClientSessionManager( self ):
@ -246,6 +250,11 @@ class Controller( object ):
return True
def IShouldRegularlyUpdate( self, window ):
return True
def ModelIsShutdown( self ):
return HG.model_shutdown