RepoMirrors
/
hydrus
mirror of https://github.com/hydrusnetwork/hydrus.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Version 262

This commit is contained in:
Hydrus Network Developer 2017-06-28 15:23:21 -05:00
parent 99c3b2fbea
commit 39fc17268f
49 changed files with 2488 additions and 1011 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

32
help/changelog.html
View File

@ -8,6 +8,38 @@
<div class="content">
<h3>changelog</h3>
<ul>
<li><h3>version 262</h3></li>
<ul>
<li>added apng support!</li>
<li>sessions will now append from the default insertion index (e.g. right of current page)</li>
<li>sessions will now load in the correct order if the default insertion index is to the left of the current page!</li>
<li>fixed an issue with custom 'favorites' gelbooru parsers</li>
<li>manage parents/siblings dialogs now load their larger datasets on a thread, after initialising</li>
<li>sped up how some default thumbnails are generated on client boot</li>
<li>sped up how resized thumbnails are generated</li>
<li>refactored ffmpeg video parsing to be simpler and easier to maintain</li>
<li>improved some ffmpeg parsing to be much faster</li>
<li>ffmpeg can now parse video that reports no duration</li>
<li>fixed some more bad video framerate parsing</li>
<li>the 4chan/8chan thread watcher will now always generate https urls (even if you enter an http thread url)</li>
<li>pixiv will now produce https urls</li>
<li>the known url file status system will now check both http and https versions of an url it is given</li>
<li>refactored some core elements to simplify common controller requests</li>
<li>started on some database migration help and ui--should be finished next week</li>
<li>flushed out network engine a little</li>
<li>more refactoring and simplification of new network engine object coupling</li>
<li>flushed out bandwidth manager</li>
<li>misc logic tweaks in bandwidth management</li>
<li>wrote a network session manager</li>
<li>wrote a new class for identifying all future 'network contexts'</li>
<li>employed NetworkContext everywhere in the new engine, massively simplifying several things</li>
<li>wrote the first batch of tests for my new network engine and its network jobs</li>
<li>wrote tests for the bandwidth rules object, including covering the new short time delta support</li>
<li>improved reliability and speed of bandwidth unit tests</li>
<li>wrote tests for the new domain-based bandwidth manager</li>
<li>misc improvements</li>
<li>more misc improvements</li>
</ul>
<li><h3>version 261</h3></li>
<ul>
<li>wrote a new manage dialog for urls</li>

56
help/database_migration.html Normal file
View File

@ -0,0 +1,56 @@
<html>
<head>
<title>migrating the database</title>
<link href="hydrus.ico" rel="shortcut icon" />
<link href="style.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div class="content">
<h3>the hydrus database</h3>
<p>A hydrus client consists of three components:</p>
<ol>
<li>
<b>the software installation</b>
<p>This is the part that comes with the installer or extract release, with the executable and dlls and a handful of resource folders. It doesn't store any of your settings--it just knows how to present a database as a nice application. If you just run the client executable straight, it looks in its 'db' subdirectory for a database, and if one is not found, it creates a new one. If it sees a database running at a lower version than itself, it will update the database before booting it.</p>
<p>It doesn't really matter where you put this. An SSD will load it marginally quicker the first time, but you probably won't notice. If you run it without command-line parameters, it will try to write to its own directory (to create the initial database), so if you mean to run it like that, it should not be in a protected place like <i>Program Files</i>.</p>
</li>
<li>
<b>the actual database</b>
<p>The client stores all its preferences and currently open tabs and knowledge <i>about</i> files--like file size and resolution, tags, ratings, inbox status, and so on and so on--in a handful of SQLite database files, defaulting to <i>install_dir/db</i>. Depending on the size of your client, these might total 1MB in size or be as much as 10GB.</p>
<p>In order to perform a search or to fetch or process tags, the client has to interact with these files in many small bursts, which means it is best if these files are on a drive with low latency. An SSD is ideal, but a regularly-defragged HDD with a reasonable amount of free space also works well.</p>
</li>
<li>
<b>your media files</b>
<p>All of your jpegs and webms and so on (and their thumbnails) are stored in a single complicated directory that is by default at <i>install_dir/db/client_files</i>. All the files are named by their hash and stored in efficient hash-based subdirectories. In general, it is not navigable by humans, but it works very well for the fast access from a giant pool of files the client needs to do to manage your media.</p>
<p>Thumbnails tend to be fetched dozens at a time, so it is, again, ideal if they are stored on an SSD. Your regular media files--which on many clients total hundreds of GB--are usually fetched one at a time for human consumption and do not benefit from the expensive low-latency of an SSD. They are best stored on a cheap HDD, and, if desired, also work well when stored across a network file system.</p>
</li>
</ol>
<h3>these components can be put on different drives</h3>
<p>Although an initial install will keep these parts together, it is possible to run the database on a fast drive but keep your files in cheap slow storage. And if you have a very large collection, you can even spread your files across multiple drives. It is not very technically difficult, but I do not recommend it for new users.</p>
<h3>pulling them apart</h3>
<p><b class="warning">As always, I recommend creating a backup before you try any of this, just in case it goes wrong.</b></p>
<p>If you have multiple drives and would like to spread your database across them, please do not move the folders around yourself--the database has an internal 'knowledge' of where it thinks its thumbnail and file folders are, and if you move them while it is closed, it will throw 'missing path' errors as soon as it boots. The internal hydrus logic of relative and absolute paths is not always obvious, so it is easy to make mistakes, even if you think you know what you are doing. Instead, please do it through the gui:</p>
<p>Go <i>database->migration</i>, giving you this dialog:</p>
<p><img src="db_migration.png" /></p>
<p class="warning">***Talk about what portable means.***</p>
<p>These operations are simple, but they can take some time to complete.</p>
<p>If you move your database, the program will have to shut down first. Before you boot up again, you will have to create a new program shortcut:</p>
<h3>informing the software that the database has moved</h3>
<p>A straight call to the client executable will look for a database in <i>install_dir/db</i>. If one is not found, it will create one. So, if you move your database and then try to run the client again, it will try to create a new empty database in the previous location!</p>
<p>So, pass it a command line argument, like so:</p>
<ul>
<li>client -d="D:\media\my_hydrus_database"</li>
<li><i>--or--</i></li>
<li>client --db_dir="G:\misc documents\New Folder (3)\DO NOT ENTER"</li>
</ul>
<p>And it will instead use the given path. You can use any path that is valid in your system, but I would not advise using network locations and so on, as the database works best with some clever device locking calls these interfaces may not provide.</p>
<p>Rather than typing the path out in a terminal every time you want to launch your external database, create a new shortcut with the argument in. Something like this:</p>
<p><img src="db_migration_shortcut.png" /></p>
<p>Note that an install with an 'external' database no longer needs access to write to its own path, so you can store it anywhere you like. If you move it, just double-check your shortcuts are still good and you are done.</p>
<h3>finally</h3>
<p>Now you have a new database in one or more new locations, make sure to update your backup routine to follow these new paths!</p>
<h3>p.s. running multiple clients</h3>
<p>Since you now know how to tell the software about an external database, you can, if you like, run multiple clients from the same install (and if you previously had multiple install folders, now you can now just use the one). Just make multiple shortcuts to the same client executable but with different database directories. They can run at the same time. You'll save yourself a little memory and update-hassle.</p>
</div>
</body>
</html>

BIN
help/db_migration_shortcut.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 20 KiB

25
include/ClientCaches.py
View File

@ -306,7 +306,7 @@ class ClientFilesManager( object ):
try:
thumbnail_resized = HydrusFileHandling.GenerateThumbnail( full_size_path, thumbnail_dimensions )
thumbnail_resized = HydrusFileHandling.GenerateThumbnailFromStaticImage( full_size_path, thumbnail_dimensions )
except:
@ -321,7 +321,7 @@ class ClientFilesManager( object ):
self._GenerateFullSizeThumbnail( hash )
thumbnail_resized = HydrusFileHandling.GenerateThumbnail( full_size_path, thumbnail_dimensions )
thumbnail_resized = HydrusFileHandling.GenerateThumbnailFromStaticImage( full_size_path, thumbnail_dimensions )
resized_path = self._GenerateExpectedResizedThumbnailPath( hash )
@ -1457,7 +1457,7 @@ class LocalBooruCache( object ):
def _RefreshShares( self ):
self._local_booru_service = self._controller.GetServicesManager().GetService( CC.LOCAL_BOORU_SERVICE_KEY )
self._local_booru_service = self._controller.services_manager.GetService( CC.LOCAL_BOORU_SERVICE_KEY )
self._keys_to_infos = {}
@ -1572,7 +1572,7 @@ class HydrusSessionManager( object ):
# session key expired or not found
service = self._controller.GetServicesManager().GetService( service_key )
service = self._controller.services_manager.GetService( service_key )
( response_gumpf, cookies ) = service.Request( HC.GET, 'session_key', return_cookies = True )
@ -1740,7 +1740,6 @@ class ThumbnailCache( object ):
cache_size = options[ 'thumbnail_cache_size' ]
self._data_cache = DataCache( self._controller, cache_size )
self._client_files_manager = self._controller.GetClientFilesManager()
self._lock = threading.Lock()
@ -1783,11 +1782,11 @@ class ThumbnailCache( object ):
if full_size:
path = self._client_files_manager.GetFullSizeThumbnailPath( hash )
path = self._controller.client_files_manager.GetFullSizeThumbnailPath( hash )
else:
path = self._client_files_manager.GetResizedThumbnailPath( hash )
path = self._controller.client_files_manager.GetResizedThumbnailPath( hash )
except HydrusExceptions.FileMissingException as e:
@ -1803,11 +1802,11 @@ class ThumbnailCache( object ):
if full_size:
path = self._client_files_manager.GetFullSizeThumbnailPath( hash )
path = self._controller.client_files_manager.GetFullSizeThumbnailPath( hash )
else:
path = self._client_files_manager.GetResizedThumbnailPath( hash )
path = self._controller.client_files_manager.GetResizedThumbnailPath( hash )
except HydrusExceptions.FileMissingException:
@ -1826,7 +1825,7 @@ class ThumbnailCache( object ):
try:
self._client_files_manager.RegenerateResizedThumbnail( hash )
self._controller.client_files_manager.RegenerateResizedThumbnail( hash )
try:
@ -1861,7 +1860,7 @@ class ThumbnailCache( object ):
if too_large or ( too_small and not small_original_image ):
self._client_files_manager.RegenerateResizedThumbnail( hash )
self._controller.client_files_manager.RegenerateResizedThumbnail( hash )
hydrus_bitmap = ClientRendering.GenerateHydrusBitmap( path )
@ -1906,7 +1905,7 @@ class ThumbnailCache( object ):
options = self._controller.GetOptions()
thumbnail = HydrusFileHandling.GenerateThumbnail( path, options[ 'thumbnail_dimensions' ] )
thumbnail = HydrusFileHandling.GenerateThumbnailFromStaticImage( path, options[ 'thumbnail_dimensions' ] )
with open( temp_path, 'wb' ) as f:
@ -3260,7 +3259,7 @@ class WebSessionManagerClient( object ):
form_fields[ 'password' ] = password
form_fields[ 'captcha' ] = ''
form_fields[ 'g_recaptcha_response' ] = ''
form_fields[ 'return_to' ] = 'http://www.pixiv.net'
form_fields[ 'return_to' ] = 'https://www.pixiv.net'
form_fields[ 'lang' ] = 'en'
form_fields[ 'post_key' ] = post_key
form_fields[ 'source' ] = 'pc'

11
include/ClientConstants.py
View File

@ -195,6 +195,7 @@ media_viewer_capabilities = {}
media_viewer_capabilities[ HC.IMAGE_JPEG ] = static_full_support
media_viewer_capabilities[ HC.IMAGE_PNG ] = static_full_support
media_viewer_capabilities[ HC.IMAGE_APNG ] = animated_full_support
media_viewer_capabilities[ HC.IMAGE_GIF ] = animated_full_support
if HC.PLATFORM_WINDOWS:
@ -240,10 +241,16 @@ NEW_PAGE_GOES_FAR_RIGHT = 3
new_page_goes_string_lookup = {}
new_page_goes_string_lookup[ NEW_PAGE_GOES_FAR_LEFT ] = 'go far left'
new_page_goes_string_lookup[ NEW_PAGE_GOES_LEFT_OF_CURRENT ] = 'go left of current page'
new_page_goes_string_lookup[ NEW_PAGE_GOES_RIGHT_OF_CURRENT ] = 'go right of current page'
new_page_goes_string_lookup[ NEW_PAGE_GOES_LEFT_OF_CURRENT ] = 'go left of current page tab'
new_page_goes_string_lookup[ NEW_PAGE_GOES_RIGHT_OF_CURRENT ] = 'go right of current page tab'
new_page_goes_string_lookup[ NEW_PAGE_GOES_FAR_RIGHT ] = 'go far right'
NETWORK_CONTEXT_GLOBAL = 0
NETWORK_CONTEXT_HYDRUS = 1
NETWORK_CONTEXT_DOMAIN = 1
NETWORK_CONTEXT_DOWNLOADER = 1
NETWORK_CONTEXT_SUBSCRIPTION = 1
SHORTCUT_MODIFIER_CTRL = 0
SHORTCUT_MODIFIER_ALT = 1
SHORTCUT_MODIFIER_SHIFT = 2

44
include/ClientController.py
View File

@ -57,6 +57,9 @@ class Controller( HydrusController.HydrusController ):
self._previously_idle = False
self._idle_started = None
self.client_files_manager = None
self.services_manager = None
def _InitDB( self ):
@ -391,13 +394,13 @@ class Controller( HydrusController.HydrusController ):
stop_time = HydrusData.GetNow() + ( self._options[ 'idle_shutdown_max_minutes' ] * 60 )
self._client_files_manager.Rebalance( partial = False, stop_time = stop_time )
self.client_files_manager.Rebalance( partial = False, stop_time = stop_time )
self.MaintainDB( stop_time = stop_time )
if not self._options[ 'pause_repo_sync' ]:
services = self.GetServicesManager().GetServices( HC.REPOSITORIES )
services = self.services_manager.GetServices( HC.REPOSITORIES )
for service in services:
@ -488,11 +491,6 @@ class Controller( HydrusController.HydrusController ):
raise NotImplementedError()
def GetClientFilesManager( self ):
return self._client_files_manager
def GetClientSessionManager( self ):
return self._client_session_manager
@ -503,7 +501,10 @@ class Controller( HydrusController.HydrusController ):
return self._shortcuts_manager.GetCommand( shortcut_names, shortcut )
def GetGUI( self ): return self._gui
def GetGUI( self ):
return self._gui
def GetOptions( self ):
@ -515,11 +516,6 @@ class Controller( HydrusController.HydrusController ):
return self._new_options
def GetServicesManager( self ):
return self._services_manager
def GoodTimeToDoForegroundWork( self ):
if self._gui:
@ -534,9 +530,9 @@ class Controller( HydrusController.HydrusController ):
def InitClientFilesManager( self ):
self._client_files_manager = ClientCaches.ClientFilesManager( self )
self.client_files_manager = ClientCaches.ClientFilesManager( self )
missing_locations = self._client_files_manager.GetMissing()
missing_locations = self.client_files_manager.GetMissing()
while len( missing_locations ) > 0:
@ -548,9 +544,9 @@ class Controller( HydrusController.HydrusController ):
if dlg.ShowModal() == wx.ID_OK:
self._client_files_manager = ClientCaches.ClientFilesManager( self )
self.client_files_manager = ClientCaches.ClientFilesManager( self )
missing_locations = self._client_files_manager.GetMissing()
missing_locations = self.client_files_manager.GetMissing()
else:
@ -568,7 +564,7 @@ class Controller( HydrusController.HydrusController ):
HydrusController.HydrusController.InitModel( self )
self._services_manager = ClientCaches.ServicesManager( self )
self.services_manager = ClientCaches.ServicesManager( self )
self._options = self.Read( 'options' )
self._new_options = self.Read( 'serialisable', HydrusSerialisable.SERIALISABLE_TYPE_CLIENT_OPTIONS )
@ -769,7 +765,7 @@ class Controller( HydrusController.HydrusController ):
self.pub( 'splash_set_status_text', 'fattening service info' )
services = self.GetServicesManager().GetServices()
services = self.services_manager.GetServices()
for service in services:
@ -870,7 +866,7 @@ class Controller( HydrusController.HydrusController ):
def RefreshServices( self ):
self._services_manager.RefreshServices()
self.services_manager.RefreshServices()
def ResetIdleTimer( self ):
@ -885,7 +881,7 @@ class Controller( HydrusController.HydrusController ):
def RestartBooru( self ):
service = self.GetServicesManager().GetService( CC.LOCAL_BOORU_SERVICE_KEY )
service = self.services_manager.GetService( CC.LOCAL_BOORU_SERVICE_KEY )
port = service.GetPort()
@ -1028,7 +1024,7 @@ class Controller( HydrusController.HydrusController ):
with HG.dirty_object_lock:
dirty_services = [ service for service in self._services_manager.GetServices() if service.IsDirty() ]
dirty_services = [ service for service in self.services_manager.GetServices() if service.IsDirty() ]
if len( dirty_services ) > 0:
@ -1043,7 +1039,7 @@ class Controller( HydrusController.HydrusController ):
self.WriteSynchronous( 'update_services', services )
self._services_manager.RefreshServices()
self.services_manager.RefreshServices()
@ -1120,7 +1116,7 @@ class Controller( HydrusController.HydrusController ):
return True
services = self.GetServicesManager().GetServices( HC.REPOSITORIES )
services = self.services_manager.GetServices( HC.REPOSITORIES )
for service in services:

40
include/ClientDB.py
View File

@ -1423,7 +1423,7 @@ class DB( HydrusDB.HydrusDB ):
hash_ids = [ hash_id for ( hash_id, ) in self._c.execute( 'SELECT hash_id FROM shape_maintenance_phash_regen;' ) ]
client_files_manager = self._controller.GetClientFilesManager()
client_files_manager = self._controller.client_files_manager
total_done_previously = total_num_hash_ids_in_cache - len( hash_ids )
@ -2470,7 +2470,7 @@ class DB( HydrusDB.HydrusDB ):
missing_count = 0
deletee_hash_ids = []
client_files_manager = self._controller.GetClientFilesManager()
client_files_manager = self._controller.client_files_manager
for ( i, ( hash_id, mime ) ) in enumerate( info ):
@ -2933,7 +2933,7 @@ class DB( HydrusDB.HydrusDB ):
deletable_file_hash_ids = hash_ids.difference( potentially_pending_upload_hash_ids )
client_files_manager = self._controller.GetClientFilesManager()
client_files_manager = self._controller.client_files_manager
if len( deletable_file_hash_ids ) > 0:
@ -5438,7 +5438,7 @@ class DB( HydrusDB.HydrusDB ):
needed_hashes = []
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
for hash_id in needed_hash_ids:
@ -6060,18 +6060,30 @@ class DB( HydrusDB.HydrusDB ):
def _GetURLStatus( self, url ):
result = self._c.execute( 'SELECT hash_id FROM urls WHERE url = ?;', ( url, ) ).fetchone()
search_urls = [ url ]
if result is None:
if url.startswith( 'http://' ):
return ( CC.STATUS_NEW, None )
search_urls.append( 'https://' + url[7:] )
else:
elif url.startswith( 'https://' ):
( hash_id, ) = result
search_urls.append( 'http://' + url[8:] )
return self._GetHashIdStatus( hash_id )
for search_url in search_urls:
result = self._c.execute( 'SELECT hash_id FROM urls WHERE url = ?;', ( search_url, ) ).fetchone()
if result is not None:
( hash_id, ) = result
return self._GetHashIdStatus( hash_id )
return ( CC.STATUS_NEW, None )
def _GetWebSessions( self ):
@ -6217,7 +6229,7 @@ class DB( HydrusDB.HydrusDB ):
timestamp = HydrusData.GetNow()
client_files_manager = self._controller.GetClientFilesManager()
client_files_manager = self._controller.client_files_manager
if mime in HC.MIMES_WITH_THUMBNAILS:
@ -6314,7 +6326,7 @@ class DB( HydrusDB.HydrusDB ):
num_frames = None
num_words = None
client_files_manager = self._controller.GetClientFilesManager()
client_files_manager = self._controller.client_files_manager
# lockless because this db call is made by the locked client files manager
client_files_manager.LocklessAddFileFromString( update_hash, mime, update_network_string )
@ -7610,7 +7622,7 @@ class DB( HydrusDB.HydrusDB ):
num_updates_done = 0
client_files_manager = self._controller.GetClientFilesManager()
client_files_manager = self._controller.client_files_manager
select_statement = 'SELECT hash_id FROM files_info WHERE mime = ' + str( HC.APPLICATION_HYDRUS_UPDATE_DEFINITIONS ) + ' AND hash_id IN %s;'
@ -7830,7 +7842,7 @@ class DB( HydrusDB.HydrusDB ):
self._controller.pub( 'message', job_key )
client_files_manager = self._controller.GetClientFilesManager()
client_files_manager = self._controller.client_files_manager
num_to_do = len( hashes )

12
include/ClientDaemons.py
View File

@ -62,7 +62,7 @@ def DAEMONDownloadFiles( controller ):
if num_downloads > 0:
client_files_manager = controller.GetClientFilesManager()
client_files_manager = controller.client_files_manager
successful_hashes = set()
@ -89,7 +89,7 @@ def DAEMONDownloadFiles( controller ):
try:
service = controller.GetServicesManager().GetService( service_key )
service = controller.services_manager.GetService( service_key )
except:
@ -233,7 +233,7 @@ def DAEMONMaintainTrash( controller ):
def DAEMONRebalanceClientFiles( controller ):
controller.GetClientFilesManager().Rebalance()
controller.client_files_manager.Rebalance()
def DAEMONSaveDirtyObjects( controller ):
@ -241,7 +241,7 @@ def DAEMONSaveDirtyObjects( controller ):
def DAEMONSynchroniseAccounts( controller ):
services = controller.GetServicesManager().GetServices( HC.RESTRICTED_SERVICES )
services = controller.services_manager.GetServices( HC.RESTRICTED_SERVICES )
for service in services:
@ -254,7 +254,7 @@ def DAEMONSynchroniseRepositories( controller ):
if not options[ 'pause_repo_sync' ]:
services = controller.GetServicesManager().GetServices( HC.REPOSITORIES )
services = controller.services_manager.GetServices( HC.REPOSITORIES )
for service in services:
@ -306,7 +306,7 @@ def DAEMONUPnP( controller ):
return # This IGD probably doesn't support UPnP, so don't spam the user with errors they can't fix!
services = controller.GetServicesManager().GetServices( ( HC.LOCAL_BOORU, ) )
services = controller.services_manager.GetServices( ( HC.LOCAL_BOORU, ) )
for service in services:

15
include/ClientData.py
View File

@ -126,7 +126,7 @@ def ConvertServiceKeysToContentUpdatesToPrettyString( service_keys_to_content_up
if len( content_updates ) > 0:
name = HG.client_controller.GetServicesManager().GetName( service_key )
name = HG.client_controller.services_manager.GetName( service_key )
locations.add( name )
@ -347,7 +347,7 @@ def GetSortChoices( add_namespaces_and_ratings = True ):
sort_choices.extend( HC.options[ 'sort_by' ] )
service_keys = HG.client_controller.GetServicesManager().GetServiceKeys( ( HC.LOCAL_RATING_LIKE, HC.LOCAL_RATING_NUMERICAL ) )
service_keys = HG.client_controller.services_manager.GetServiceKeys( ( HC.LOCAL_RATING_LIKE, HC.LOCAL_RATING_NUMERICAL ) )
for service_key in service_keys:
@ -679,7 +679,7 @@ class ApplicationCommand( HydrusSerialisable.SerialisableBase ):
components.append( '"' + HydrusData.ToUnicode( value ) + '"' )
components.append( 'for' )
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
if services_manager.ServiceExists( service_key ):
@ -899,6 +899,7 @@ class ClientOptions( HydrusSerialisable.SerialisableBase ):
self._dictionary[ 'media_view' ][ HC.IMAGE_JPEG ] = ( CC.MEDIA_VIEWER_ACTION_SHOW_AS_NORMAL, CC.MEDIA_VIEWER_ACTION_SHOW_AS_NORMAL, image_zoom_info )
self._dictionary[ 'media_view' ][ HC.IMAGE_PNG ] = ( CC.MEDIA_VIEWER_ACTION_SHOW_AS_NORMAL, CC.MEDIA_VIEWER_ACTION_SHOW_AS_NORMAL, image_zoom_info )
self._dictionary[ 'media_view' ][ HC.IMAGE_APNG ] = ( CC.MEDIA_VIEWER_ACTION_SHOW_AS_NORMAL, CC.MEDIA_VIEWER_ACTION_SHOW_AS_NORMAL, gif_zoom_info )
self._dictionary[ 'media_view' ][ HC.IMAGE_GIF ] = ( CC.MEDIA_VIEWER_ACTION_SHOW_AS_NORMAL, CC.MEDIA_VIEWER_ACTION_SHOW_AS_NORMAL, gif_zoom_info )
if HC.PLATFORM_WINDOWS:
@ -1566,7 +1567,7 @@ class DuplicateActionOptions( HydrusSerialisable.SerialisableBase ):
if HG.client_controller.IsBooted():
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
self._tag_service_actions = [ ( service_key, action, tag_censor ) for ( service_key, action, tag_censor ) in self._tag_service_actions if services_manager.ServiceExists( service_key ) and services_manager.GetServiceType( service_key ) in ( HC.LOCAL_TAG, HC.TAG_REPOSITORY ) ]
self._rating_service_actions = [ ( service_key, action ) for ( service_key, action ) in self._rating_service_actions if services_manager.ServiceExists( service_key ) and services_manager.GetServiceType( service_key ) in ( HC.LOCAL_RATING_LIKE, HC.LOCAL_RATING_NUMERICAL ) ]
@ -1664,7 +1665,7 @@ class DuplicateActionOptions( HydrusSerialisable.SerialisableBase ):
service_keys_to_content_updates = {}
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
for ( service_key, action, tag_censor ) in self._tag_service_actions:
@ -1992,7 +1993,7 @@ class ImportTagOptions( HydrusSerialisable.SerialisableBase ):
if HG.client_controller.IsBooted():
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
test_func = services_manager.ServiceExists
@ -2263,7 +2264,7 @@ class Shortcuts( HydrusSerialisable.SerialisableBaseNamed ):
# this never stored mouse actions, so skip
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
for ( modifier, key, ( serialisable_service_key, data ) ) in serialisable_keyboard_actions:

61
include/ClientDownloading.py
View File

@ -137,7 +137,7 @@ def GetImageboardFileURL( thread_url, filename, ext ):
if is_4chan:
return 'http://i.4cdn.org/' + board + '/' + filename + ext
return 'https://i.4cdn.org/' + board + '/' + filename + ext
elif is_8chan:
@ -151,7 +151,7 @@ def GetImageboardFileURL( thread_url, filename, ext ):
try:
html_url = 'http://8ch.net/' + board + '/res/' + thread_id + '.html'
html_url = 'https://8ch.net/' + board + '/res/' + thread_id + '.html'
response = ClientNetworking.RequestsGet( html_url )
@ -182,7 +182,7 @@ def GetImageboardFileURL( thread_url, filename, ext ):
media_host = _8CHAN_BOARDS_TO_MEDIA_HOSTS[ board ]
return 'http://' + media_host + '/' + board + '/src/' + filename + ext
return 'https://' + media_host + '/' + board + '/src/' + filename + ext
@ -194,18 +194,18 @@ def GetImageboardThreadJSONURL( thread_url ):
is_8chan = '8ch.net' in host
# 4chan
# http://a.4cdn.org/asp/thread/382059.json
# https://a.4cdn.org/asp/thread/382059.json
# 8chan
# http://8ch.net/v/res/406061.json
# https://8ch.net/v/res/406061.json
if is_4chan:
return 'http://a.4cdn.org/' + board + '/thread/' + thread_id + '.json'
return 'https://a.4cdn.org/' + board + '/thread/' + thread_id + '.json'
elif is_8chan:
return 'http://8ch.net/' + board + '/res/' + thread_id + '.json'
return 'https://8ch.net/' + board + '/res/' + thread_id + '.json'
def GetSoup( html ):
@ -245,7 +245,7 @@ def THREADDownloadURL( job_key, url, url_string ):
job_key.SetVariable( 'popup_text_1', 'importing' )
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
( result, hash ) = client_files_manager.ImportFile( temp_path )
@ -336,7 +336,7 @@ def THREADDownloadURLs( job_key, urls, title ):
job_key.SetVariable( 'popup_text_2', 'importing' )
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
( result, hash ) = client_files_manager.ImportFile( temp_path )
@ -879,27 +879,34 @@ class GalleryBooru( Gallery ):
# https://gelbooru.com/redirect.php?s=Ly9nZWxib29ydS5jb20vaW5kZXgucGhwP3BhZ2U9cG9zdCZzPXZpZXcmaWQ9MzY5NDEyMg==
try:
if 'redirect.php' in bad_url:
encoded_location = bad_url.split( '?s=' )[1]
try:
encoded_location = bad_url.split( '?s=' )[1]
location = encoded_location.decode( 'base64' )
url = urlparse.urljoin( bad_url, location )
urls.append( url )
except Exception as e:
HydrusData.ShowText( 'gelbooru parsing problem!' )
HydrusData.ShowException( e )
url = ClientNetworking.RequestsGetRedirectURL( bad_url, session )
urls.append( url )
time.sleep( 0.5 )
location = encoded_location.decode( 'base64' )
url = urlparse.urljoin( bad_url, location )
else:
urls.append( url )
except Exception as e:
HydrusData.ShowText( 'gelbooru parsing problem!' )
HydrusData.ShowException( e )
url = ClientNetworking.RequestsGetRedirectURL( bad_url, session )
urls.append( url )
time.sleep( 0.5 )
@ -1752,7 +1759,7 @@ class GalleryPixivArtistID( GalleryPixiv ):
artist_id = query
gallery_url = 'http://www.pixiv.net/member_illust.php?type=illust&id=' + str( artist_id )
gallery_url = 'https://www.pixiv.net/member_illust.php?type=illust&id=' + str( artist_id )
return gallery_url + '&p=' + str( page_index + 1 )
@ -1763,7 +1770,7 @@ class GalleryPixivTag( GalleryPixiv ):
tag = query
gallery_url = 'http://www.pixiv.net/search.php?word=' + urllib.quote( HydrusData.ToByteString( tag ), '' ) + '&s_mode=s_tag_full&order=date_d'
gallery_url = 'https://www.pixiv.net/search.php?word=' + urllib.quote( HydrusData.ToByteString( tag ), '' ) + '&s_mode=s_tag_full&order=date_d'
return gallery_url + '&p=' + str( page_index + 1 )

2
include/ClientExporting.py
View File

@ -287,7 +287,7 @@ class ExportFolder( HydrusSerialisable.SerialisableBaseNamed ):
sync_filenames = set()
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
num_copied = 0

122
include/ClientGUI.py
View File

@ -242,7 +242,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
subject_account_key = dlg.GetValue().decode( 'hex' )
service = self._controller.GetServicesManager().GetService( service_key )
service = self._controller.services_manager.GetService( service_key )
response = service.Request( HC.GET, 'account_info', { 'subject_account_key' : subject_account_key } )
@ -280,7 +280,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
def _AppendGUISession( self, name ):
def do_it( session ):
def do_it( session, starting_index ):
try:
@ -291,6 +291,8 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
wx.CallAfter( self._notebook.Disable )
forced_insertion_index = starting_index
for ( page_name, management_controller, initial_hashes ) in session.IteratePages():
try:
@ -315,7 +317,9 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
initial_media_results = []
wx.CallAfter( self._NewPage, page_name, management_controller, initial_media_results = initial_media_results )
wx.CallAfter( self._NewPage, page_name, management_controller, initial_media_results = initial_media_results, forced_insertion_index = forced_insertion_index )
forced_insertion_index += 1
except Exception as e:
@ -358,7 +362,9 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
return
self._controller.CallToThread( do_it, session )
starting_index = self._GetDefaultPageInsertionIndex()
self._controller.CallToThread( do_it, session, starting_index )
def _AutoRepoSetup( self ):
@ -395,7 +401,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
file_repo.SetCredentials( credentials )
all_services = self._controller.GetServicesManager().GetServices()
all_services = self._controller.services_manager.GetServices()
all_services.append( tag_repo )
all_services.append( file_repo )
@ -540,13 +546,13 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
#
all_services = list( self._controller.GetServicesManager().GetServices() )
all_services = list( self._controller.services_manager.GetServices() )
all_services.append( admin_service )
self._controller.SetServices( all_services )
admin_service = self._controller.GetServicesManager().GetService( admin_service_key ) # let's refresh it
admin_service = self._controller.services_manager.GetService( admin_service_key ) # let's refresh it
HydrusData.ShowText( 'Admin service initialised.' )
@ -607,7 +613,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
started = HydrusData.GetNow()
service = self._controller.GetServicesManager().GetService( service_key )
service = self._controller.services_manager.GetService( service_key )
service.Request( HC.POST, 'backup' )
@ -660,7 +666,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
def _CheckFileIntegrity( self ):
client_files_manager = self._controller.GetClientFilesManager()
client_files_manager = self._controller.client_files_manager
message = 'This will go through all the files the database thinks it has and check that they actually exist. Any files that are missing will be deleted from the internal record.'
message += os.linesep * 2
@ -727,7 +733,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
text = 'What would you like to do with the orphaned files? Note that all orphaned thumbnails will be deleted.'
client_files_manager = self._controller.GetClientFilesManager()
client_files_manager = self._controller.client_files_manager
with ClientGUIDialogs.DialogYesNo( self, text, title = 'Choose what do to with the orphans.', yes_label = 'move them somewhere', no_label = 'delete them' ) as dlg_2:
@ -928,7 +934,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
def _DeletePending( self, service_key ):
service = self._controller.GetServicesManager().GetService( service_key )
service = self._controller.services_manager.GetService( service_key )
with ClientGUIDialogs.DialogYesNo( self, 'Are you sure you want to delete the pending data for ' + service.GetName() + '?' ) as dlg:
@ -972,7 +978,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
hash = dlg.GetValue().decode( 'hex' )
service = self._controller.GetServicesManager().GetService( service_key )
service = self._controller.services_manager.GetService( service_key )
with wx.BusyCursor(): response = service.Request( HC.GET, 'ip', { 'hash' : hash } )
@ -1156,7 +1162,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
search_menu = wx.Menu()
services = self._controller.GetServicesManager().GetServices()
services = self._controller.services_manager.GetServices()
petition_permissions = [ ( content_type, HC.PERMISSION_ACTION_OVERRULE ) for content_type in HC.REPOSITORY_CONTENT_TYPES ]
@ -1303,7 +1309,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
for ( service_key, info ) in nums_pending.items():
service = self._controller.GetServicesManager().GetService( service_key )
service = self._controller.services_manager.GetService( service_key )
service_type = service.GetServiceType()
name = service.GetName()
@ -1369,8 +1375,8 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
def services():
tag_services = self._controller.GetServicesManager().GetServices( ( HC.TAG_REPOSITORY, ) )
file_services = self._controller.GetServicesManager().GetServices( ( HC.FILE_REPOSITORY, ) )
tag_services = self._controller.services_manager.GetServices( ( HC.TAG_REPOSITORY, ) )
file_services = self._controller.services_manager.GetServices( ( HC.FILE_REPOSITORY, ) )
submenu = wx.Menu()
@ -1388,10 +1394,10 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
repository_admin_permissions = [ ( HC.CONTENT_TYPE_ACCOUNTS, HC.PERMISSION_ACTION_CREATE ), ( HC.CONTENT_TYPE_ACCOUNTS, HC.PERMISSION_ACTION_OVERRULE ), ( HC.CONTENT_TYPE_ACCOUNT_TYPES, HC.PERMISSION_ACTION_OVERRULE ) ]
repositories = self._controller.GetServicesManager().GetServices( HC.REPOSITORIES )
repositories = self._controller.services_manager.GetServices( HC.REPOSITORIES )
admin_repositories = [ service for service in repositories if True in ( service.HasPermission( content_type, action ) for ( content_type, action ) in repository_admin_permissions ) ]
servers_admin = self._controller.GetServicesManager().GetServices( ( HC.SERVER_ADMIN, ) )
servers_admin = self._controller.services_manager.GetServices( ( HC.SERVER_ADMIN, ) )
server_admins = [ service for service in servers_admin if service.HasPermission( HC.CONTENT_TYPE_SERVICES, HC.PERMISSION_ACTION_OVERRULE ) ]
if len( admin_repositories ) > 0 or len( server_admins ) > 0:
@ -1582,6 +1588,37 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
with ClientGUIDialogs.DialogGenerateNewAccounts( self, service_key ) as dlg: dlg.ShowModal()
def _GetDefaultPageInsertionIndex( self ):
new_page_goes = self._new_options.GetInteger( 'default_new_page_goes' )
current_index = self._notebook.GetSelection()
if current_index == wx.NOT_FOUND:
new_page_goes = CC.NEW_PAGE_GOES_FAR_LEFT
if new_page_goes == CC.NEW_PAGE_GOES_FAR_LEFT:
insertion_index = 0
elif new_page_goes == CC.NEW_PAGE_GOES_LEFT_OF_CURRENT:
insertion_index = current_index
elif new_page_goes == CC.NEW_PAGE_GOES_RIGHT_OF_CURRENT:
insertion_index = current_index + 1
elif new_page_goes == CC.NEW_PAGE_GOES_FAR_RIGHT:
insertion_index = self._notebook.GetPageCount()
return insertion_index
def _ImportFiles( self, paths = None ):
if paths is None: paths = []
@ -1931,7 +1968,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
return
service = self._controller.GetServicesManager().GetService( service_key )
service = self._controller.services_manager.GetService( service_key )
with ClientGUIDialogs.DialogTextEntry( self, 'Enter the account key for the account to be modified.' ) as dlg:
@ -1987,7 +2024,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
def _NewPage( self, page_name, management_controller, initial_media_results = None ):
def _NewPage( self, page_name, management_controller, initial_media_results = None, forced_insertion_index = None ):
self._controller.ResetIdleTimer()
self._controller.ResetPageChangeTimer()
@ -1999,39 +2036,22 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
page = ClientGUIPages.Page( self._notebook, self._controller, management_controller, initial_media_results )
if self._next_new_page_index is None:
if forced_insertion_index is None:
new_page_goes = self._new_options.GetInteger( 'default_new_page_goes' )
current_index = self._notebook.GetSelection()
if current_index == wx.NOT_FOUND:
if self._next_new_page_index is None:
new_page_goes = CC.NEW_PAGE_GOES_FAR_LEFT
insertion_index = self._GetDefaultPageInsertionIndex()
if new_page_goes == CC.NEW_PAGE_GOES_FAR_LEFT:
else:
insertion_index = 0
insertion_index = self._next_new_page_index
elif new_page_goes == CC.NEW_PAGE_GOES_LEFT_OF_CURRENT:
insertion_index = current_index
elif new_page_goes == CC.NEW_PAGE_GOES_RIGHT_OF_CURRENT:
insertion_index = current_index + 1
elif new_page_goes == CC.NEW_PAGE_GOES_FAR_RIGHT:
insertion_index = self._notebook.GetPageCount()
self._next_new_page_index = None
else:
insertion_index = self._next_new_page_index
self._next_new_page_index = None
insertion_index = forced_insertion_index
self._notebook.InsertPage( insertion_index, page, page_name, select = True )
@ -2095,7 +2115,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
management_controller = ClientGUIManagement.CreateManagementControllerPetitions( service_key )
service = self._controller.GetServicesManager().GetService( service_key )
service = self._controller.services_manager.GetService( service_key )
page_name = service.GetName() + ' petitions'
@ -2113,7 +2133,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
tag_service_key = new_options.GetKey( 'default_tag_service_search_page' )
if not self._controller.GetServicesManager().ServiceExists( tag_service_key ):
if not self._controller.services_manager.ServiceExists( tag_service_key ):
tag_service_key = CC.COMBINED_TAG_SERVICE_KEY
@ -2265,7 +2285,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
if dlg.ShowModal() == wx.ID_YES:
self._controller.CallToThread( self._controller.GetClientFilesManager().Rebalance, partial = False )
self._controller.CallToThread( self._controller.client_files_manager.Rebalance, partial = False )
@ -2368,7 +2388,7 @@ class FrameGUI( ClientGUITopLevelWindows.FrameThatResizes ):
def _RegenerateThumbnails( self ):
client_files_manager = self._controller.GetClientFilesManager()
client_files_manager = self._controller.client_files_manager
text = 'This will rebuild all your thumbnails from the original files. You probably only want to do this if you experience thumbnail errors. If you have a lot of files, it will take some time. A popup message will show its progress.'
text += os.linesep * 2
@ -2556,7 +2576,7 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
def _StartIPFSDownload( self ):
ipfs_services = self._controller.GetServicesManager().GetServices( ( HC.IPFS, ) )
ipfs_services = self._controller.services_manager.GetServices( ( HC.IPFS, ) )
if len( ipfs_services ) > 0:
@ -2811,7 +2831,7 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
def _THREADUploadPending( self, service_key ):
service = self._controller.GetServicesManager().GetService( service_key )
service = self._controller.services_manager.GetService( service_key )
service_name = service.GetName()
service_type = service.GetServiceType()
@ -2869,7 +2889,7 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
media_result = result
client_files_manager = self._controller.GetClientFilesManager()
client_files_manager = self._controller.client_files_manager
hash = media_result.GetHash()
mime = media_result.GetMime()

12
include/ClientGUIACDropdown.py
View File

@ -521,9 +521,9 @@ class AutoCompleteDropdownTags( AutoCompleteDropdown ):
self._current_matches = []
file_service = HG.client_controller.GetServicesManager().GetService( self._file_service_key )
file_service = HG.client_controller.services_manager.GetService( self._file_service_key )
tag_service = HG.client_controller.GetServicesManager().GetService( self._tag_service_key )
tag_service = HG.client_controller.services_manager.GetService( self._tag_service_key )
self._file_repo_button = ClientGUICommon.BetterButton( self._dropdown_window, file_service.GetName(), self.FileButtonHit )
self._file_repo_button.SetMinSize( ( 20, -1 ) )
@ -541,7 +541,7 @@ class AutoCompleteDropdownTags( AutoCompleteDropdown ):
self._file_service_key = file_service_key
file_service = HG.client_controller.GetServicesManager().GetService( self._file_service_key )
file_service = HG.client_controller.services_manager.GetService( self._file_service_key )
name = file_service.GetName()
@ -561,7 +561,7 @@ class AutoCompleteDropdownTags( AutoCompleteDropdown ):
self._dropdown_list.SetTagService( self._tag_service_key )
tag_service = tag_service = HG.client_controller.GetServicesManager().GetService( self._tag_service_key )
tag_service = tag_service = HG.client_controller.services_manager.GetService( self._tag_service_key )
name = tag_service.GetName()
@ -594,7 +594,7 @@ class AutoCompleteDropdownTags( AutoCompleteDropdown ):
def FileButtonHit( self ):
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
services = []
@ -626,7 +626,7 @@ class AutoCompleteDropdownTags( AutoCompleteDropdown ):
def TagButtonHit( self ):
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
services = []

28
include/ClientGUICanvas.py
View File

@ -423,7 +423,7 @@ class Animation( wx.Window ):
hash = self._media.GetHash()
mime = self._media.GetMime()
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
path = client_files_manager.GetFilePath( hash, mime )
@ -1270,7 +1270,7 @@ class Canvas( wx.Window ):
if self._current_media is not None:
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
paths = [ client_files_manager.GetFilePath( self._current_media.GetHash(), self._current_media.GetMime() ) ]
@ -1282,7 +1282,7 @@ class Canvas( wx.Window ):
if self._current_media is not None:
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
path = client_files_manager.GetFilePath( self._current_media.GetHash(), self._current_media.GetMime() )
@ -1556,7 +1556,7 @@ class Canvas( wx.Window ):
return
if len( HG.client_controller.GetServicesManager().GetServices( HC.RATINGS_SERVICES ) ) > 0:
if len( HG.client_controller.services_manager.GetServices( HC.RATINGS_SERVICES ) ) > 0:
with ClientGUIDialogsManage.DialogManageRatings( self, ( self._current_media, ) ) as dlg:
@ -1636,7 +1636,7 @@ class Canvas( wx.Window ):
hash = self._current_media.GetHash()
mime = self._current_media.GetMime()
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
path = client_files_manager.GetFilePath( hash, mime )
@ -1736,7 +1736,7 @@ class Canvas( wx.Window ):
try:
service = HG.client_controller.GetServicesManager().GetService( service_key )
service = HG.client_controller.services_manager.GetService( service_key )
except HydrusExceptions.DataMissing:
@ -2439,7 +2439,7 @@ class CanvasPanel( Canvas ):
if self._current_media is not None:
services = HG.client_controller.GetServicesManager().GetServices()
services = HG.client_controller.services_manager.GetServices()
locations_manager = self._current_media.GetLocationsManager()
@ -2686,7 +2686,7 @@ class CanvasWithDetails( Canvas ):
# ratings
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
like_services = services_manager.GetServices( ( HC.LOCAL_RATING_LIKE, ), randomised = False )
@ -2847,7 +2847,7 @@ class CanvasWithHovers( CanvasWithDetails ):
self._hover_commands = self._GenerateHoverTopFrame()
self._hover_tags = ClientGUIHoverFrames.FullscreenHoverFrameTags( self, self._canvas_key )
ratings_services = HG.client_controller.GetServicesManager().GetServices( ( HC.RATINGS_SERVICES ) )
ratings_services = HG.client_controller.services_manager.GetServices( ( HC.RATINGS_SERVICES ) )
if len( ratings_services ) > 0:
@ -4708,7 +4708,7 @@ class CanvasMediaListBrowser( CanvasMediaListNavigable ):
if self._current_media is not None:
services = HG.client_controller.GetServicesManager().GetServices()
services = HG.client_controller.services_manager.GetServices()
local_ratings_services = [ service for service in services if service.GetServiceType() in ( HC.LOCAL_RATING_LIKE, HC.LOCAL_RATING_NUMERICAL ) ]
@ -4983,7 +4983,7 @@ class MediaContainer( wx.Window ):
raise Exception( 'Failed to initialise the flash window' )
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
self._media_window.movie = client_files_manager.GetFilePath( self._media.GetHash(), HC.APPLICATION_FLASH )
@ -5422,7 +5422,7 @@ class EmbedButton( wx.Window ):
hash = self._media.GetHash()
thumbnail_path = HG.client_controller.GetClientFilesManager().GetFullSizeThumbnailPath( hash )
thumbnail_path = HG.client_controller.client_files_manager.GetFullSizeThumbnailPath( hash )
self._thumbnail_bmp = ClientRendering.GenerateHydrusBitmap( thumbnail_path ).GetWxBitmap()
@ -5450,7 +5450,7 @@ class OpenExternallyPanel( wx.Panel ):
hash = self._media.GetHash()
thumbnail_path = HG.client_controller.GetClientFilesManager().GetFullSizeThumbnailPath( hash )
thumbnail_path = HG.client_controller.client_files_manager.GetFullSizeThumbnailPath( hash )
bmp = ClientRendering.GenerateHydrusBitmap( thumbnail_path ).GetWxBitmap()
@ -5480,7 +5480,7 @@ class OpenExternallyPanel( wx.Panel ):
hash = self._media.GetHash()
mime = self._media.GetMime()
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
path = client_files_manager.GetFilePath( hash, mime )

8
include/ClientGUICommon.py
View File

@ -482,7 +482,7 @@ class CheckboxCollect( wx.combo.ComboCtrl ):
text_and_data_tuples = list( [ ( namespace, ( 'namespace', namespace ) ) for namespace in text_and_data_tuples ] )
text_and_data_tuples.sort()
ratings_services = HG.client_controller.GetServicesManager().GetServices( ( HC.LOCAL_RATING_LIKE, HC.LOCAL_RATING_NUMERICAL ) )
ratings_services = HG.client_controller.services_manager.GetServices( ( HC.LOCAL_RATING_LIKE, HC.LOCAL_RATING_NUMERICAL ) )
for ratings_service in ratings_services:
@ -598,7 +598,7 @@ class ChoiceSort( BetterChoice ):
self._page_key = page_key
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
sort_choices = ClientData.GetSortChoices( add_namespaces_and_ratings = add_namespaces_and_ratings )
@ -2539,7 +2539,7 @@ class RatingLikeCanvas( RatingLike ):
self._current_media = None
self._rating_state = None
service = HG.client_controller.GetServicesManager().GetService( service_key )
service = HG.client_controller.services_manager.GetService( service_key )
name = service.GetName()
@ -2648,7 +2648,7 @@ class RatingNumerical( wx.Window ):
self._service_key = service_key
self._service = HG.client_controller.GetServicesManager().GetService( self._service_key )
self._service = HG.client_controller.services_manager.GetService( self._service_key )
self._num_stars = self._service.GetNumStars()
self._allow_zero = self._service.AllowZero()

28
include/ClientGUIDialogs.py
View File

@ -61,7 +61,7 @@ def SelectServiceKey( service_types = HC.ALL_SERVICES, service_keys = None, unal
if service_keys is None:
services = HG.client_controller.GetServicesManager().GetServices( service_types )
services = HG.client_controller.services_manager.GetServices( service_types )
service_keys = [ service.GetServiceKey() for service in services ]
@ -83,7 +83,7 @@ def SelectServiceKey( service_types = HC.ALL_SERVICES, service_keys = None, unal
else:
services = { HG.client_controller.GetServicesManager().GetService( service_key ) for service_key in service_keys }
services = { HG.client_controller.services_manager.GetService( service_key ) for service_key in service_keys }
list_of_tuples = [ ( service.GetName(), service.GetServiceKey() ) for service in services ]
@ -372,7 +372,7 @@ class DialogGenerateNewAccounts( Dialog ):
self._num.SetValue( 1 )
service = HG.client_controller.GetServicesManager().GetService( service_key )
service = HG.client_controller.services_manager.GetService( service_key )
response = service.Request( HC.GET, 'account_types' )
@ -439,7 +439,7 @@ class DialogGenerateNewAccounts( Dialog ):
expires = HydrusData.GetNow() + lifetime
service = HG.client_controller.GetServicesManager().GetService( self._service_key )
service = HG.client_controller.services_manager.GetService( self._service_key )
try:
@ -570,7 +570,7 @@ class DialogInputFileSystemPredicates( Dialog ):
elif predicate_type == HC.PREDICATE_TYPE_SYSTEM_RATING:
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
ratings_services = services_manager.GetServices( ( HC.LOCAL_RATING_LIKE, HC.LOCAL_RATING_NUMERICAL ) )
@ -778,7 +778,7 @@ class DialogInputLocalBooruShare( Dialog ):
def EventCopyExternalShareURL( self, event ):
self._service = HG.client_controller.GetServicesManager().GetService( CC.LOCAL_BOORU_SERVICE_KEY )
self._service = HG.client_controller.services_manager.GetService( CC.LOCAL_BOORU_SERVICE_KEY )
external_ip = HydrusNATPunch.GetExternalIP() # eventually check for optional host replacement here
@ -796,7 +796,7 @@ class DialogInputLocalBooruShare( Dialog ):
def EventCopyInternalShareURL( self, event ):
self._service = HG.client_controller.GetServicesManager().GetService( CC.LOCAL_BOORU_SERVICE_KEY )
self._service = HG.client_controller.services_manager.GetService( CC.LOCAL_BOORU_SERVICE_KEY )
internal_ip = '127.0.0.1'
@ -1606,7 +1606,7 @@ class DialogModifyAccounts( Dialog ):
Dialog.__init__( self, parent, 'modify account' )
self._service = HG.client_controller.GetServicesManager().GetService( service_key )
self._service = HG.client_controller.services_manager.GetService( service_key )
self._subject_identifiers = list( subject_identifiers )
#
@ -1828,7 +1828,7 @@ class DialogPageChooser( Dialog ):
self.SetInitialSize( ( 420, 210 ) )
self._services = HG.client_controller.GetServicesManager().GetServices()
self._services = HG.client_controller.services_manager.GetServices()
repository_petition_permissions = [ ( content_type, HC.PERMISSION_ACTION_OVERRULE ) for content_type in HC.REPOSITORY_CONTENT_TYPES ]
@ -1862,7 +1862,7 @@ class DialogPageChooser( Dialog ):
elif entry_type in ( 'page_query', 'page_petitions' ):
name = HG.client_controller.GetServicesManager().GetService( obj ).GetName()
name = HG.client_controller.services_manager.GetService( obj ).GetName()
button.SetLabelText( name )
@ -2119,7 +2119,7 @@ class DialogPathsToTags( Dialog ):
#
services = HG.client_controller.GetServicesManager().GetServices( ( HC.TAG_REPOSITORY, ) )
services = HG.client_controller.services_manager.GetServices( ( HC.TAG_REPOSITORY, ) )
for service in services:
@ -3393,7 +3393,7 @@ class DialogSetupExport( Dialog ):
self._tags_box = ClientGUICommon.StaticBoxSorterForListBoxTags( self, 'files\' tags' )
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
self._neighbouring_txt_tag_service_keys = services_manager.FilterValidServiceKeys( new_options.GetKeyList( 'default_neighbouring_txt_tag_service_keys' ) )
@ -3580,7 +3580,7 @@ class DialogSetupExport( Dialog ):
terms = ClientExporting.ParseExportPhrase( pattern )
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
self._export.Disable()
@ -3650,7 +3650,7 @@ class DialogSetupExport( Dialog ):
def EventExportTagTxtsChanged( self, event ):
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
tag_services = services_manager.GetServices( HC.TAG_SERVICES )

211
include/ClientGUIDialogsManage.py
View File

@ -2534,7 +2534,7 @@ class DialogManageImportFoldersEdit( ClientGUIDialogs.Dialog ):
self._txt_parse_st = wx.StaticText( self._tag_box, label = '' )
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
self._txt_parse_tag_service_keys = services_manager.FilterValidServiceKeys( txt_parse_tag_service_keys )
@ -2730,7 +2730,7 @@ class DialogManageImportFoldersEdit( ClientGUIDialogs.Dialog ):
def _RefreshTxtParseText( self ):
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
services = [ services_manager.GetService( service_key ) for service_key in self._txt_parse_tag_service_keys ]
@ -2772,7 +2772,7 @@ class DialogManageImportFoldersEdit( ClientGUIDialogs.Dialog ):
def EventEditTxtParsing( self, event ):
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
tag_services = services_manager.GetServices( HC.TAG_SERVICES )
@ -3042,8 +3042,8 @@ class DialogManageRatings( ClientGUIDialogs.Dialog ):
#
like_services = HG.client_controller.GetServicesManager().GetServices( ( HC.LOCAL_RATING_LIKE, ), randomised = False )
numerical_services = HG.client_controller.GetServicesManager().GetServices( ( HC.LOCAL_RATING_NUMERICAL, ), randomised = False )
like_services = HG.client_controller.services_manager.GetServices( ( HC.LOCAL_RATING_LIKE, ), randomised = False )
numerical_services = HG.client_controller.services_manager.GetServices( ( HC.LOCAL_RATING_NUMERICAL, ), randomised = False )
self._panels = []
@ -3502,7 +3502,7 @@ class DialogManageTagCensorship( ClientGUIDialogs.Dialog ):
#
services = HG.client_controller.GetServicesManager().GetServices( ( HC.COMBINED_TAG, HC.TAG_REPOSITORY, HC.LOCAL_TAG ) )
services = HG.client_controller.services_manager.GetServices( ( HC.COMBINED_TAG, HC.TAG_REPOSITORY, HC.LOCAL_TAG ) )
for service in services:
@ -3668,7 +3668,7 @@ class DialogManageTagParents( ClientGUIDialogs.Dialog ):
#
services = HG.client_controller.GetServicesManager().GetServices( ( HC.TAG_REPOSITORY, ) )
services = HG.client_controller.services_manager.GetServices( ( HC.TAG_REPOSITORY, ) )
for service in services:
@ -3769,15 +3769,9 @@ class DialogManageTagParents( ClientGUIDialogs.Dialog ):
if service_key != CC.LOCAL_TAG_SERVICE_KEY:
self._service = HG.client_controller.GetServicesManager().GetService( service_key )
self._service = HG.client_controller.services_manager.GetService( service_key )
self._original_statuses_to_pairs = HG.client_controller.Read( 'tag_parents', service_key )
self._current_statuses_to_pairs = collections.defaultdict( set )
self._current_statuses_to_pairs.update( { key : set( value ) for ( key, value ) in self._original_statuses_to_pairs.items() } )
self._pairs_to_reasons = {}
self._tag_parents = ClientGUICommon.SaneListCtrl( self, 250, [ ( '', 30 ), ( 'child', 160 ), ( 'parent', -1 ) ] )
@ -3791,7 +3785,10 @@ class DialogManageTagParents( ClientGUIDialogs.Dialog ):
expand_parents = True
self._child_input = ClientGUIACDropdown.AutoCompleteDropdownTagsWrite( self, self.EnterChildren, expand_parents, CC.LOCAL_FILE_SERVICE_KEY, service_key )
self._child_input.Disable()
self._parent_input = ClientGUIACDropdown.AutoCompleteDropdownTagsWrite( self, self.EnterParents, expand_parents, CC.LOCAL_FILE_SERVICE_KEY, service_key )
self._parent_input.Disable()
self._add = wx.Button( self, label = 'add' )
self._add.Bind( wx.EVT_BUTTON, self.EventAddButton )
@ -3799,36 +3796,9 @@ class DialogManageTagParents( ClientGUIDialogs.Dialog ):
#
petitioned_pairs = set( self._original_statuses_to_pairs[ HC.CONTENT_STATUS_PETITIONED ] )
for ( status, pairs ) in self._original_statuses_to_pairs.items():
if status != HC.CONTENT_STATUS_DELETED:
sign = HydrusData.ConvertStatusToPrefix( status )
for ( child, parent ) in pairs:
if status == HC.CONTENT_STATUS_CURRENT and ( child, parent ) in petitioned_pairs:
continue
self._tag_parents.Append( ( sign, child, parent ), ( status, child, parent ) )
self._tag_parents.SortListItems( 2 )
if tags is not None:
self.EnterChildren( tags )
#
intro = 'Files with a tag on the left will also be given the tag on the right.'
self._status_st = ClientGUICommon.BetterStaticText( self, u'initialising\u2026' )
tags_box = wx.BoxSizer( wx.HORIZONTAL )
@ -3842,7 +3812,7 @@ class DialogManageTagParents( ClientGUIDialogs.Dialog ):
vbox = wx.BoxSizer( wx.VERTICAL )
vbox.AddF( ClientGUICommon.BetterStaticText( self, intro ), CC.FLAGS_EXPAND_PERPENDICULAR )
vbox.AddF( self._status_st, CC.FLAGS_EXPAND_PERPENDICULAR )
vbox.AddF( self._tag_parents, CC.FLAGS_EXPAND_BOTH_WAYS )
vbox.AddF( self._add, CC.FLAGS_LONE_BUTTON )
vbox.AddF( tags_box, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
@ -3850,6 +3820,10 @@ class DialogManageTagParents( ClientGUIDialogs.Dialog ):
self.SetSizer( vbox )
#
HG.client_controller.CallToThread( self.THREADInitialise, tags )
def _AddPairs( self, children, parent ):
@ -4231,6 +4205,57 @@ class DialogManageTagParents( ClientGUIDialogs.Dialog ):
else: self._parent_input.SetFocus()
def THREADInitialise( self, tags ):
def wx_code():
if not self:
return
self._status_st.SetLabelText( 'Files with a tag on the left will also be given the tag on the right.' )
self._child_input.Enable()
self._parent_input.Enable()
petitioned_pairs = set( self._original_statuses_to_pairs[ HC.CONTENT_STATUS_PETITIONED ] )
for ( status, pairs ) in self._original_statuses_to_pairs.items():
if status != HC.CONTENT_STATUS_DELETED:
sign = HydrusData.ConvertStatusToPrefix( status )
for ( child, parent ) in pairs:
if status == HC.CONTENT_STATUS_CURRENT and ( child, parent ) in petitioned_pairs:
continue
self._tag_parents.Append( ( sign, child, parent ), ( status, child, parent ) )
self._tag_parents.SortListItems( 2 )
if tags is not None:
self.EnterChildren( tags )
self._original_statuses_to_pairs = HG.client_controller.Read( 'tag_parents', self._service_key )
self._current_statuses_to_pairs = collections.defaultdict( set )
self._current_statuses_to_pairs.update( { key : set( value ) for ( key, value ) in self._original_statuses_to_pairs.items() } )
wx.CallAfter( wx_code )
class DialogManageTagSiblings( ClientGUIDialogs.Dialog ):
@ -4256,7 +4281,7 @@ class DialogManageTagSiblings( ClientGUIDialogs.Dialog ):
self._tag_repositories.AddPage( name, name, page )
services = HG.client_controller.GetServicesManager().GetServices( ( HC.TAG_REPOSITORY, ) )
services = HG.client_controller.services_manager.GetServices( ( HC.TAG_REPOSITORY, ) )
for service in services:
@ -4351,15 +4376,9 @@ class DialogManageTagSiblings( ClientGUIDialogs.Dialog ):
if self._service_key != CC.LOCAL_TAG_SERVICE_KEY:
self._service = HG.client_controller.GetServicesManager().GetService( service_key )
self._service = HG.client_controller.services_manager.GetService( service_key )
self._original_statuses_to_pairs = HG.client_controller.Read( 'tag_siblings', service_key )
self._current_statuses_to_pairs = collections.defaultdict( set )
self._current_statuses_to_pairs.update( { key : set( value ) for ( key, value ) in self._original_statuses_to_pairs.items() } )
self._pairs_to_reasons = {}
self._current_new = None
@ -4375,7 +4394,10 @@ class DialogManageTagSiblings( ClientGUIDialogs.Dialog ):
expand_parents = False
self._old_input = ClientGUIACDropdown.AutoCompleteDropdownTagsWrite( self, self.EnterOlds, expand_parents, CC.LOCAL_FILE_SERVICE_KEY, service_key )
self._old_input.Disable()
self._new_input = ClientGUIACDropdown.AutoCompleteDropdownTagsWrite( self, self.SetNew, expand_parents, CC.LOCAL_FILE_SERVICE_KEY, service_key )
self._new_input.Disable()
self._add = wx.Button( self, label = 'add' )
self._add.Bind( wx.EVT_BUTTON, self.EventAddButton )
@ -4383,36 +4405,7 @@ class DialogManageTagSiblings( ClientGUIDialogs.Dialog ):
#
petitioned_pairs = set( self._original_statuses_to_pairs[ HC.CONTENT_STATUS_PETITIONED ] )
for ( status, pairs ) in self._original_statuses_to_pairs.items():
if status != HC.CONTENT_STATUS_DELETED:
sign = HydrusData.ConvertStatusToPrefix( status )
for ( old, new ) in pairs:
if status == HC.CONTENT_STATUS_CURRENT and ( old, new ) in petitioned_pairs:
continue
self._tag_siblings.Append( ( sign, old, new ), ( status, old, new ) )
self._tag_siblings.SortListItems( 2 )
if tags is not None:
self.EnterOlds( tags )
#
intro = 'Tags on the left will be replaced by those on the right.'
self._status_st = ClientGUICommon.BetterStaticText( self, u'initialising\u2026' )
new_sibling_box = wx.BoxSizer( wx.VERTICAL )
@ -4432,7 +4425,7 @@ class DialogManageTagSiblings( ClientGUIDialogs.Dialog ):
vbox = wx.BoxSizer( wx.VERTICAL )
vbox.AddF( ClientGUICommon.BetterStaticText( self, intro ), CC.FLAGS_EXPAND_PERPENDICULAR )
vbox.AddF( self._status_st, CC.FLAGS_EXPAND_PERPENDICULAR )
vbox.AddF( self._tag_siblings, CC.FLAGS_EXPAND_BOTH_WAYS )
vbox.AddF( self._add, CC.FLAGS_LONE_BUTTON )
vbox.AddF( text_box, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
@ -4440,6 +4433,9 @@ class DialogManageTagSiblings( ClientGUIDialogs.Dialog ):
self.SetSizer( vbox )
#
HG.client_controller.CallToThread( self.THREADInitialise, tags )
def _AddPairs( self, olds, new ):
@ -4893,6 +4889,57 @@ class DialogManageTagSiblings( ClientGUIDialogs.Dialog ):
else: self._new_input.SetFocus()
def THREADInitialise( self, tags ):
def wx_code():
if not self:
return
self._status_st.SetLabelText( 'Tags on the left will be replaced by those on the right.' )
self._old_input.Enable()
self._new_input.Enable()
petitioned_pairs = set( self._original_statuses_to_pairs[ HC.CONTENT_STATUS_PETITIONED ] )
for ( status, pairs ) in self._original_statuses_to_pairs.items():
if status != HC.CONTENT_STATUS_DELETED:
sign = HydrusData.ConvertStatusToPrefix( status )
for ( old, new ) in pairs:
if status == HC.CONTENT_STATUS_CURRENT and ( old, new ) in petitioned_pairs:
continue
self._tag_siblings.Append( ( sign, old, new ), ( status, old, new ) )
self._tag_siblings.SortListItems( 2 )
if tags is not None:
self.EnterOlds( tags )
self._original_statuses_to_pairs = HG.client_controller.Read( 'tag_siblings', self._service_key )
self._current_statuses_to_pairs = collections.defaultdict( set )
self._current_statuses_to_pairs.update( { key : set( value ) for ( key, value ) in self._original_statuses_to_pairs.items() } )
wx.CallAfter( wx_code )
class DialogManageUPnP( ClientGUIDialogs.Dialog ):

4
include/ClientGUIHoverFrames.py
View File

@ -724,7 +724,7 @@ class FullscreenHoverFrameTopRight( FullscreenHoverFrame ):
like_hbox.AddF( ( 16, 16 ), CC.FLAGS_EXPAND_BOTH_WAYS )
like_services = HG.client_controller.GetServicesManager().GetServices( ( HC.LOCAL_RATING_LIKE, ), randomised = False )
like_services = HG.client_controller.services_manager.GetServices( ( HC.LOCAL_RATING_LIKE, ), randomised = False )
for service in like_services:
@ -739,7 +739,7 @@ class FullscreenHoverFrameTopRight( FullscreenHoverFrame ):
vbox.AddF( like_hbox, CC.FLAGS_EXPAND_SIZER_BOTH_WAYS )
numerical_services = HG.client_controller.GetServicesManager().GetServices( ( HC.LOCAL_RATING_NUMERICAL, ), randomised = False )
numerical_services = HG.client_controller.services_manager.GetServices( ( HC.LOCAL_RATING_NUMERICAL, ), randomised = False )
for service in numerical_services:

14
include/ClientGUIManagement.py
View File

@ -136,7 +136,7 @@ def CreateManagementControllerImportURLs():
def CreateManagementControllerPetitions( petition_service_key ):
petition_service = HG.client_controller.GetServicesManager().GetService( petition_service_key )
petition_service = HG.client_controller.services_manager.GetService( petition_service_key )
petition_service_type = petition_service.GetServiceType()
@ -394,13 +394,13 @@ def GenerateDumpMultipartFormDataCTAndBody( fields ):
def EventRefreshCaptcha( self, event ):
javascript_string = self._controller.DoHTTP( HC.GET, 'http://www.google.com/recaptcha/api/challenge?k=' + self._captcha_key )
javascript_string = self._controller.DoHTTP( HC.GET, 'https://www.google.com/recaptcha/api/challenge?k=' + self._captcha_key )
( trash, rest ) = javascript_string.split( 'challenge : \'', 1 )
( self._captcha_challenge, trash ) = rest.split( '\'', 1 )
jpeg = self._controller.DoHTTP( HC.GET, 'http://www.google.com/recaptcha/api/image?c=' + self._captcha_challenge )
jpeg = self._controller.DoHTTP( HC.GET, 'https://www.google.com/recaptcha/api/image?c=' + self._captcha_challenge )
( os_file_handle, temp_path ) = HydrusPaths.GetTempPath()
@ -562,7 +562,7 @@ class ManagementController( HydrusSerialisable.SerialisableBase ):
if 'file_service' in self._keys:
if not HG.client_controller.GetServicesManager().ServiceExists( self._keys[ 'file_service' ] ):
if not HG.client_controller.services_manager.ServiceExists( self._keys[ 'file_service' ] ):
self._keys[ 'file_service' ] = CC.COMBINED_LOCAL_FILE_SERVICE_KEY
@ -864,7 +864,7 @@ class ManagementPanelDuplicateFilter( ManagementPanel ):
def _FileDomainButtonHit( self ):
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
services = []
@ -945,7 +945,7 @@ class ManagementPanelDuplicateFilter( ManagementPanel ):
self._management_controller.SetKey( 'duplicate_filter_file_domain', service_key )
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
service = services_manager.GetService( service_key )
@ -2879,7 +2879,7 @@ class ManagementPanelPetitions( ManagementPanel ):
ManagementPanel.__init__( self, parent, page, controller, management_controller )
self._service = self._controller.GetServicesManager().GetService( self._petition_service_key )
self._service = self._controller.services_manager.GetService( self._petition_service_key )
self._can_ban = self._service.HasPermission( HC.CONTENT_TYPE_ACCOUNTS, HC.PERMISSION_ACTION_OVERRULE )
service_type = self._service.GetServiceType()

36
include/ClientGUIMedia.py
View File

@ -40,7 +40,7 @@ ID_TIMER_ANIMATION = wx.NewId()
def AddServiceKeyLabelsToMenu( menu, service_keys, phrase ):
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
if len( service_keys ) == 1:
@ -68,7 +68,7 @@ def AddServiceKeyLabelsToMenu( menu, service_keys, phrase ):
def AddServiceKeysToMenu( event_handler, menu, service_keys, phrase, description, callable ):
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
if len( service_keys ) == 1:
@ -164,7 +164,7 @@ class MediaPanel( ClientMedia.ListeningMediaList, wx.ScrolledWindow ):
def _CopyFilesToClipboard( self ):
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
hashes = self._GetSelectedHashes( discriminant = CC.DISCRIMINANT_LOCAL, ordered = True )
@ -233,7 +233,7 @@ class MediaPanel( ClientMedia.ListeningMediaList, wx.ScrolledWindow ):
display_media = self._focussed_media.GetDisplayMedia()
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
path = client_files_manager.GetFilePath( display_media.GetHash(), display_media.GetMime() )
@ -244,7 +244,7 @@ class MediaPanel( ClientMedia.ListeningMediaList, wx.ScrolledWindow ):
media_results = self.GenerateMediaResults( discriminant = CC.DISCRIMINANT_LOCAL, selected_media = set( self._selected_media ) )
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
paths = []
@ -266,7 +266,7 @@ class MediaPanel( ClientMedia.ListeningMediaList, wx.ScrolledWindow ):
( filename, ) = HG.client_controller.Read( 'service_filenames', service_key, { hash } )
service = HG.client_controller.GetServicesManager().GetService( service_key )
service = HG.client_controller.services_manager.GetService( service_key )
if service.GetServiceType() == HC.IPFS:
@ -282,7 +282,7 @@ class MediaPanel( ClientMedia.ListeningMediaList, wx.ScrolledWindow ):
prefix = ''
service = HG.client_controller.GetServicesManager().GetService( service_key )
service = HG.client_controller.services_manager.GetService( service_key )
if service.GetServiceType() == HC.IPFS:
@ -483,7 +483,7 @@ class MediaPanel( ClientMedia.ListeningMediaList, wx.ScrolledWindow ):
hash = display_media.GetHash()
mime = display_media.GetMime()
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
path = client_files_manager.GetFilePath( hash, mime )
@ -843,7 +843,7 @@ class MediaPanel( ClientMedia.ListeningMediaList, wx.ScrolledWindow ):
if len( self._selected_media ) > 0:
if len( HG.client_controller.GetServicesManager().GetServices( HC.RATINGS_SERVICES ) ) > 0:
if len( HG.client_controller.services_manager.GetServices( HC.RATINGS_SERVICES ) ) > 0:
flat_media = self._GetSelectedFlatMedia()
@ -929,7 +929,7 @@ class MediaPanel( ClientMedia.ListeningMediaList, wx.ScrolledWindow ):
hash = self._focussed_media.GetHash()
mime = self._focussed_media.GetMime()
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
path = client_files_manager.GetFilePath( hash, mime )
@ -949,7 +949,7 @@ class MediaPanel( ClientMedia.ListeningMediaList, wx.ScrolledWindow ):
hash = self._focussed_media.GetHash()
mime = self._focussed_media.GetMime()
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
path = client_files_manager.GetFilePath( hash, mime )
@ -966,7 +966,7 @@ class MediaPanel( ClientMedia.ListeningMediaList, wx.ScrolledWindow ):
if hashes is not None and len( hashes ) > 0:
remote_service = HG.client_controller.GetServicesManager().GetService( remote_service_key )
remote_service = HG.client_controller.services_manager.GetService( remote_service_key )
service_type = remote_service.GetServiceType()
@ -1454,7 +1454,7 @@ class MediaPanel( ClientMedia.ListeningMediaList, wx.ScrolledWindow ):
if hashes is not None and len( hashes ) > 0:
ipfs_service = HG.client_controller.GetServicesManager().GetService( file_service_key )
ipfs_service = HG.client_controller.services_manager.GetService( file_service_key )
with ClientGUIDialogs.DialogTextEntry( self, 'Enter a note to describe this directory.' ) as dlg:
@ -1850,7 +1850,7 @@ class MediaPanelThumbnails( MediaPanel ):
if len( self._selected_media ) > 0:
services = HG.client_controller.GetServicesManager().GetServices( ( HC.LOCAL_TAG, HC.TAG_REPOSITORY, HC.COMBINED_TAG ) )
services = HG.client_controller.services_manager.GetServices( ( HC.LOCAL_TAG, HC.TAG_REPOSITORY, HC.COMBINED_TAG ) )
service_keys = [ service.GetServiceKey() for service in services ]
@ -2345,7 +2345,7 @@ class MediaPanelThumbnails( MediaPanel ):
file_data_object = wx.FileDataObject()
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
for hash in hashes:
@ -2583,7 +2583,7 @@ class MediaPanelThumbnails( MediaPanel ):
advanced_mode = new_options.GetBoolean( 'advanced_mode' )
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
thumbnail = self._GetThumbnailUnderMouse( event )
@ -2668,7 +2668,7 @@ class MediaPanelThumbnails( MediaPanel ):
multiple_selected = num_selected > 1
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
services = services_manager.GetServices()
@ -3927,7 +3927,7 @@ class Thumbnail( Selectable ):
# repo icons
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
repo_icon_x = 0

2
include/ClientGUIOptionsPanels.py
View File

@ -308,7 +308,7 @@ class OptionsPanelTags( OptionsPanel ):
self._services_vbox.Clear( True )
services = HG.client_controller.GetServicesManager().GetServices( HC.TAG_SERVICES, randomised = False )
services = HG.client_controller.services_manager.GetServices( HC.TAG_SERVICES, randomised = False )
button_id = 1

2
include/ClientGUIPanels.py
View File

@ -774,7 +774,7 @@ class ReviewServicePanel( wx.Panel ):
job_key.SetVariable( 'popup_title', 'exporting updates for ' + self._service.GetName() )
HG.client_controller.pub( 'message', job_key )
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
for ( i, update_hash ) in enumerate( update_hashes ):

6
include/ClientGUIPredicates.py
View File

@ -184,7 +184,7 @@ class PanelPredicateSystemFileService( PanelPredicateSystem ):
self._current_pending = ClientGUICommon.BetterRadioBox( self, choices = [ ( 'currently in', HC.CONTENT_STATUS_CURRENT ), ( 'pending to', HC.CONTENT_STATUS_PENDING ) ], style = wx.RA_SPECIFY_ROWS )
services = HG.client_controller.GetServicesManager().GetServices( HC.FILE_SERVICES )
services = HG.client_controller.services_manager.GetServices( HC.FILE_SERVICES )
choices = [ ( service.GetName(), service.GetServiceKey() ) for service in services ]
@ -491,7 +491,7 @@ class PanelPredicateSystemRating( PanelPredicateSystem ):
#
local_like_services = HG.client_controller.GetServicesManager().GetServices( ( HC.LOCAL_RATING_LIKE, ), randomised = False )
local_like_services = HG.client_controller.services_manager.GetServices( ( HC.LOCAL_RATING_LIKE, ), randomised = False )
self._like_checkboxes_to_info = {}
@ -523,7 +523,7 @@ class PanelPredicateSystemRating( PanelPredicateSystem ):
#
local_numerical_services = HG.client_controller.GetServicesManager().GetServices( ( HC.LOCAL_RATING_NUMERICAL, ), randomised = False )
local_numerical_services = HG.client_controller.services_manager.GetServices( ( HC.LOCAL_RATING_NUMERICAL, ), randomised = False )
self._numerical_checkboxes_to_info = {}

10
include/ClientGUIScrolledPanelsEdit.py
View File

@ -179,7 +179,7 @@ class EditDuplicateActionOptionsPanel( ClientGUIScrolledPanels.EditPanel ):
( tag_service_options, rating_service_options, delete_second_file, sync_archive, delete_both_files ) = duplicate_action_options.ToTuple()
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
for ( service_key, action, tag_censor ) in tag_service_options:
@ -265,7 +265,7 @@ class EditDuplicateActionOptionsPanel( ClientGUIScrolledPanels.EditPanel ):
existing_service_keys.add( service_key )
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
choice_tuples = []
@ -344,7 +344,7 @@ class EditDuplicateActionOptionsPanel( ClientGUIScrolledPanels.EditPanel ):
existing_service_keys.add( service_key )
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
choice_tuples = []
@ -548,7 +548,7 @@ class EditDuplicateActionOptionsPanel( ClientGUIScrolledPanels.EditPanel ):
( service_key, action ) = sort_tuple
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
service = services_manager.GetService( service_key )
@ -563,7 +563,7 @@ class EditDuplicateActionOptionsPanel( ClientGUIScrolledPanels.EditPanel ):
( service_key, action, tag_censor ) = sort_tuple
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
service = services_manager.GetService( service_key )

26
include/ClientGUIScrolledPanelsManagement.py
View File

@ -41,7 +41,7 @@ class ManageAccountTypesPanel( ClientGUIScrolledPanels.ManagePanel ):
def __init__( self, parent, service_key ):
self._admin_service = HG.client_controller.GetServicesManager().GetService( service_key )
self._admin_service = HG.client_controller.services_manager.GetService( service_key )
ClientGUIScrolledPanels.ManagePanel.__init__( self, parent )
@ -260,7 +260,7 @@ class ManageClientServicesPanel( ClientGUIScrolledPanels.ManagePanel ):
#
all_services = HG.client_controller.GetServicesManager().GetServices()
all_services = HG.client_controller.services_manager.GetServices()
for service in all_services:
@ -2440,7 +2440,7 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
rows.append( ( 'Main gui title: ', self._main_gui_title ) )
rows.append( ( 'Default session on startup: ', self._default_gui_session ) )
rows.append( ( 'By default, new pages: ', self._default_new_page_goes ) )
rows.append( ( 'By default, new page tabs: ', self._default_new_page_goes ) )
rows.append( ( 'Confirm client exit: ', self._confirm_client_exit ) )
rows.append( ( 'Confirm sending files to trash: ', self._confirm_trash ) )
rows.append( ( 'Confirm sending more than one file to archive or inbox: ', self._confirm_archive ) )
@ -2586,7 +2586,7 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
self._media_zooms.SetValue( ','.join( ( str( media_zoom ) for media_zoom in media_zooms ) ) )
mimes_in_correct_order = ( HC.IMAGE_JPEG, HC.IMAGE_PNG, HC.IMAGE_GIF, HC.APPLICATION_FLASH, HC.APPLICATION_PDF, HC.APPLICATION_HYDRUS_UPDATE_CONTENT, HC.APPLICATION_HYDRUS_UPDATE_DEFINITIONS, HC.VIDEO_AVI, HC.VIDEO_FLV, HC.VIDEO_MOV, HC.VIDEO_MP4, HC.VIDEO_MKV, HC.VIDEO_MPEG, HC.VIDEO_WEBM, HC.VIDEO_WMV, HC.AUDIO_MP3, HC.AUDIO_OGG, HC.AUDIO_FLAC, HC.AUDIO_WMA )
mimes_in_correct_order = ( HC.IMAGE_JPEG, HC.IMAGE_PNG, HC.IMAGE_APNG, HC.IMAGE_GIF, HC.APPLICATION_FLASH, HC.APPLICATION_PDF, HC.APPLICATION_HYDRUS_UPDATE_CONTENT, HC.APPLICATION_HYDRUS_UPDATE_DEFINITIONS, HC.VIDEO_AVI, HC.VIDEO_FLV, HC.VIDEO_MOV, HC.VIDEO_MP4, HC.VIDEO_MKV, HC.VIDEO_MPEG, HC.VIDEO_WEBM, HC.VIDEO_WMV, HC.AUDIO_MP3, HC.AUDIO_OGG, HC.AUDIO_FLAC, HC.AUDIO_WMA )
for mime in mimes_in_correct_order:
@ -3219,7 +3219,7 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
self._suggested_favourites_services.Append( CC.LOCAL_TAG_SERVICE_KEY, CC.LOCAL_TAG_SERVICE_KEY )
tag_services = HG.client_controller.GetServicesManager().GetServices( ( HC.TAG_REPOSITORY, ) )
tag_services = HG.client_controller.services_manager.GetServices( ( HC.TAG_REPOSITORY, ) )
for tag_service in tag_services:
@ -3280,7 +3280,7 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
self._default_tag_service_search_page.Append( 'all known tags', CC.COMBINED_TAG_SERVICE_KEY )
services = HG.client_controller.GetServicesManager().GetServices( HC.TAG_SERVICES )
services = HG.client_controller.services_manager.GetServices( HC.TAG_SERVICES )
for service in services:
@ -3528,7 +3528,7 @@ class ManageServerServicesPanel( ClientGUIScrolledPanels.ManagePanel ):
def __init__( self, parent, service_key ):
self._clientside_admin_service = HG.client_controller.GetServicesManager().GetService( service_key )
self._clientside_admin_service = HG.client_controller.services_manager.GetService( service_key )
ClientGUIScrolledPanels.ManagePanel.__init__( self, parent )
@ -4211,7 +4211,7 @@ class ManageShortcutsPanel( ClientGUIScrolledPanels.ManagePanel ):
#
services = HG.client_controller.GetServicesManager().GetServices( ( HC.LOCAL_TAG, HC.TAG_REPOSITORY, HC.LOCAL_RATING_LIKE, HC.LOCAL_RATING_NUMERICAL ) )
services = HG.client_controller.services_manager.GetServices( ( HC.LOCAL_TAG, HC.TAG_REPOSITORY, HC.LOCAL_RATING_LIKE, HC.LOCAL_RATING_NUMERICAL ) )
for service in services:
@ -4245,7 +4245,7 @@ class ManageShortcutsPanel( ClientGUIScrolledPanels.ManagePanel ):
( service_key, content_type, action, value ) = data
self._service = HG.client_controller.GetServicesManager().GetService( service_key )
self._service = HG.client_controller.services_manager.GetService( service_key )
service_name = self._service.GetName()
service_type = self._service.GetServiceType()
@ -4528,7 +4528,7 @@ class ManageShortcutsPanel( ClientGUIScrolledPanels.ManagePanel ):
service_key = self._ratings_like_service_keys.GetClientData( selection )
service = HG.client_controller.GetServicesManager().GetService( service_key )
service = HG.client_controller.services_manager.GetService( service_key )
self._current_ratings_like_service = service
@ -4542,7 +4542,7 @@ class ManageShortcutsPanel( ClientGUIScrolledPanels.ManagePanel ):
service_key = self._ratings_numerical_service_keys.GetClientData( selection )
service = HG.client_controller.GetServicesManager().GetService( service_key )
service = HG.client_controller.services_manager.GetService( service_key )
self._current_ratings_numerical_service = service
@ -5104,7 +5104,7 @@ class ManageTagsPanel( ClientGUIScrolledPanels.ManagePanel ):
#
services = HG.client_controller.GetServicesManager().GetServices( HC.TAG_SERVICES )
services = HG.client_controller.services_manager.GetServices( HC.TAG_SERVICES )
for service in services:
@ -5313,7 +5313,7 @@ class ManageTagsPanel( ClientGUIScrolledPanels.ManagePanel ):
if not self._i_am_local_tag_service:
self._service = HG.client_controller.GetServicesManager().GetService( tag_service_key )
self._service = HG.client_controller.services_manager.GetService( tag_service_key )
self._tags_box_sorter = ClientGUICommon.StaticBoxSorterForListBoxTags( self, 'tags' )

35
include/ClientGUIScrolledPanelsReview.py
View File

@ -35,7 +35,7 @@ class AdvancedContentUpdatePanel( ClientGUIScrolledPanels.ReviewPanel ):
self._service_key = service_key
self._hashes = hashes
service = HG.client_controller.GetServicesManager().GetService( self._service_key )
service = HG.client_controller.services_manager.GetService( self._service_key )
self._service_name = service.GetName()
@ -58,7 +58,7 @@ class AdvancedContentUpdatePanel( ClientGUIScrolledPanels.ReviewPanel ):
#
services = [ service for service in HG.client_controller.GetServicesManager().GetServices( HC.TAG_SERVICES ) if service.GetServiceKey() != self._service_key ]
services = [ service for service in HG.client_controller.services_manager.GetServices( HC.TAG_SERVICES ) if service.GetServiceKey() != self._service_key ]
if len( services ) > 0:
@ -323,7 +323,7 @@ class ReviewServicesPanel( ClientGUIScrolledPanels.ReviewPanel ):
listbook_dict = {}
services = self._controller.GetServicesManager().GetServices( randomised = False )
services = self._controller.services_manager.GetServices( randomised = False )
lb_to_select = None
service_type_name_to_select = None
@ -436,3 +436,32 @@ class ReviewServicesPanel( ClientGUIScrolledPanels.ReviewPanel ):
self._InitialiseServices()
class MigrateDatabasePanel( ClientGUIScrolledPanels.ReviewPanel ):
def __init__( self, parent, controller ):
self._controller = controller
ClientGUIScrolledPanels.ReviewPanel.__init__( self, parent )
# a help button, well labelled, that points to the help page
# current install dir
# current db dir
# current file dir stuff, listctrl
# location | portable | current percents: ftr | ideal percents: ftr
# move the db and all portable client_files locations (provides warning about the shortcut and lets you copy the new location)
# this will require a shutdown
# rebalance files, listctrl
# location | portable yes/no | weight | ideal percent
# every change here, if valid, is saved immediately
# store all resized thumbs in sep location
# store all full_size thumbs in sep location
# do rebalance now button, only enabled if there is work to do
# should report to a stoppable job_key panel or something. text, gauge, stop button

14
include/ClientImporting.py
View File

@ -213,7 +213,7 @@ class GalleryImport( HydrusSerialisable.SerialisableBase ):
gallery.GetFile( temp_path, url, report_hooks = [ self._file_download_hook ] )
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
( status, hash ) = client_files_manager.ImportFile( temp_path, import_file_options = self._import_file_options )
@ -723,7 +723,7 @@ class HDDImport( HydrusSerialisable.SerialisableBase ):
raise Exception( 'File failed to copy--see log for error.' )
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
( status, hash ) = client_files_manager.ImportFile( temp_path, import_file_options = self._import_file_options )
@ -1168,7 +1168,7 @@ class ImportFolder( HydrusSerialisable.SerialisableBaseNamed ):
raise Exception( 'File failed to copy--see log for error.' )
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
( status, hash ) = client_files_manager.ImportFile( temp_path, import_file_options = self._import_file_options )
@ -1434,7 +1434,7 @@ class PageOfImagesImport( HydrusSerialisable.SerialisableBase ):
HG.client_controller.DoHTTP( HC.GET, file_url, report_hooks = report_hooks, temp_path = temp_path )
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
( status, hash ) = client_files_manager.ImportFile( temp_path, import_file_options = self._import_file_options )
@ -2364,7 +2364,7 @@ class Subscription( HydrusSerialisable.SerialisableBaseNamed ):
job_key.SetVariable( 'popup_text_1', x_out_of_y + 'importing file' )
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
( status, hash ) = client_files_manager.ImportFile( temp_path, import_file_options = self._import_file_options )
@ -2858,7 +2858,7 @@ class ThreadWatcherImport( HydrusSerialisable.SerialisableBase ):
HG.client_controller.DoHTTP( HC.GET, file_url, report_hooks = report_hooks, temp_path = temp_path )
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
( status, hash ) = client_files_manager.ImportFile( temp_path, import_file_options = self._import_file_options )
@ -3299,7 +3299,7 @@ class URLsImport( HydrusSerialisable.SerialisableBase ):
HG.client_controller.DoHTTP( HC.GET, file_url, report_hooks = report_hooks, temp_path = temp_path )
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
( status, hash ) = client_files_manager.ImportFile( temp_path, import_file_options = self._import_file_options )

8
include/ClientLocalServerResources.py
View File

@ -43,7 +43,7 @@ class HydrusResourceBooruFile( HydrusResourceBooru ):
local_booru_manager.CheckFileAuthorised( share_key, hash )
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
path = client_files_manager.GetFilePath( hash )
@ -244,7 +244,7 @@ class HydrusResourceBooruThumbnail( HydrusResourceBooru ):
if mime in HC.MIMES_WITH_THUMBNAILS:
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
path = client_files_manager.GetFullSizeThumbnailPath( hash )
@ -265,7 +265,7 @@ class HydrusResourceLocalFile( HydrusServerResources.HydrusResource ):
hash = request.hydrus_args[ 'hash' ]
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
path = client_files_manager.GetFilePath( hash )
@ -280,7 +280,7 @@ class HydrusResourceLocalThumbnail( HydrusServerResources.HydrusResource ):
hash = request.hydrus_args[ 'hash' ]
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
path = client_files_manager.GetFullSizeThumbnailPath( hash )

18
include/ClientMedia.py
View File

@ -54,7 +54,7 @@ def GetDuplicateComparisonStatements( shown_media, comparison_media ):
score += 2
elif size_ratio > 1.0:
elif size_ratio > 1.05:
statements.append( 'This has a larger filesize.' )
@ -66,7 +66,7 @@ def GetDuplicateComparisonStatements( shown_media, comparison_media ):
score -= 2
elif size_ratio < 1.0:
elif size_ratio < 0.95:
statements.append( 'This has a smaller filesize.' )
@ -396,7 +396,7 @@ class LocationsManager( object ):
pending = self.GetPendingRemote()
petitioned = self.GetPetitionedRemote()
remote_services = HG.client_controller.GetServicesManager().GetServices( ( HC.FILE_REPOSITORY, HC.IPFS ) )
remote_services = HG.client_controller.services_manager.GetServices( ( HC.FILE_REPOSITORY, HC.IPFS ) )
remote_services = list( remote_services )
@ -627,7 +627,7 @@ class MediaList( object ):
namespaces_to_collect_by = [ data for ( collect_by_type, data ) in collect_by if collect_by_type == 'namespace' ]
ratings_to_collect_by = [ data for ( collect_by_type, data ) in collect_by if collect_by_type == 'rating' ]
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
for media in medias:
@ -739,7 +739,7 @@ class MediaList( object ):
elif sort_by_data in ( CC.SORT_BY_OLDEST, CC.SORT_BY_NEWEST ):
file_service = HG.client_controller.GetServicesManager().GetService( self._file_service_key )
file_service = HG.client_controller.services_manager.GetService( self._file_service_key )
file_service_type = file_service.GetServiceType()
@ -1228,7 +1228,7 @@ class MediaList( object ):
if action == HC.CONTENT_UPDATE_DELETE:
local_file_domains = HG.client_controller.GetServicesManager().GetServiceKeys( ( HC.LOCAL_FILE_DOMAIN, ) )
local_file_domains = HG.client_controller.services_manager.GetServiceKeys( ( HC.LOCAL_FILE_DOMAIN, ) )
non_trash_local_file_services = list( local_file_domains ) + [ CC.COMBINED_LOCAL_FILE_SERVICE_KEY ]
@ -1729,7 +1729,7 @@ class MediaSingleton( Media ):
timestamp = locations_manager.GetTimestamp( service_key )
service = HG.client_controller.GetServicesManager().GetService( service_key )
service = HG.client_controller.services_manager.GetService( service_key )
service_type = service.GetServiceType()
@ -1895,7 +1895,7 @@ class MediaResult( object ):
def DeletePending( self, service_key ):
service = HG.client_controller.GetServicesManager().GetService( service_key )
service = HG.client_controller.services_manager.GetService( service_key )
service_type = service.GetServiceType()
@ -1983,7 +1983,7 @@ class MediaResult( object ):
( data_type, action, row ) = content_update.ToTuple()
service = HG.client_controller.GetServicesManager().GetService( service_key )
service = HG.client_controller.services_manager.GetService( service_key )
service_type = service.GetServiceType()

699
include/ClientNetworking.py
View File

File diff suppressed because it is too large Load Diff

2
include/ClientParsing.py
View File

@ -744,7 +744,7 @@ class ParseRootFileLookup( HydrusSerialisable.SerialisableBaseNamed ):
hash = media.GetHash()
mime = media.GetMime()
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
try:

8
include/ClientRatings.py
View File

@ -188,7 +188,7 @@ def GetNumericalStateFromMedia( media, service_key ):
def GetNumericalWidth( service_key ):
service = HG.client_controller.GetServicesManager().GetService( service_key )
service = HG.client_controller.services_manager.GetService( service_key )
num_stars = service.GetNumStars()
@ -196,7 +196,7 @@ def GetNumericalWidth( service_key ):
def GetPenAndBrushColours( service_key, rating_state ):
service = HG.client_controller.GetServicesManager().GetService( service_key )
service = HG.client_controller.services_manager.GetService( service_key )
colour = service.GetColour( rating_state )
@ -209,7 +209,7 @@ def GetPenAndBrushColours( service_key, rating_state ):
def GetShape( service_key ):
service = HG.client_controller.GetServicesManager().GetService( service_key )
service = HG.client_controller.services_manager.GetService( service_key )
shape = service.GetShape()
@ -217,7 +217,7 @@ def GetShape( service_key ):
def GetStars( service_key, rating_state, rating ):
service = HG.client_controller.GetServicesManager().GetService( service_key )
service = HG.client_controller.services_manager.GetService( service_key )
allow_zero = service.AllowZero()

26
include/ClientRendering.py
View File

@ -60,7 +60,7 @@ class ImageRenderer( object ):
hash = self._media.GetHash()
mime = self._media.GetMime()
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
self._path = client_files_manager.GetFilePath( hash, mime )
@ -158,7 +158,7 @@ class RasterContainer( object ):
hash = self._media.GetHash()
mime = self._media.GetMime()
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
self._path = client_files_manager.GetFilePath( hash, mime )
@ -329,7 +329,7 @@ class RasterContainerVideo( RasterContainer ):
duration = self._media.GetDuration()
num_frames = self._media.GetNumFrames()
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
if self._media.GetMime() == HC.IMAGE_GIF:
@ -441,8 +441,14 @@ class RasterContainerVideo( RasterContainer ):
def GetDuration( self, index ):
if self._media.GetMime() == HC.IMAGE_GIF: return self._durations[ index ]
else: return self._average_frame_duration
if self._media.GetMime() == HC.IMAGE_GIF:
return self._durations[ index ]
else:
return self._average_frame_duration
def GetFrame( self, index ):
@ -546,8 +552,14 @@ class RasterContainerVideo( RasterContainer ):
def GetTotalDuration( self ):
if self._media.GetMime() == HC.IMAGE_GIF: return sum( self._durations )
else: return self._average_frame_duration * self.GetNumFrames()
if self._media.GetMime() == HC.IMAGE_GIF:
return sum( self._durations )
else:
return self._average_frame_duration * self.GetNumFrames()
def HasFrame( self, index ):

6
include/ClientSearch.py
View File

@ -296,7 +296,7 @@ class FileSearchContext( HydrusSerialisable.SerialisableBase ):
self._file_service_key = file_service_key.decode( 'hex' )
self._tag_service_key = tag_service_key.decode( 'hex' )
services_manager = HG.client_controller.GetServicesManager()
services_manager = HG.client_controller.services_manager
if not services_manager.ServiceExists( self._file_service_key ):
@ -1101,7 +1101,7 @@ class Predicate( HydrusSerialisable.SerialisableBase ):
( operator, value, service_key ) = self._value
service = HG.client_controller.GetServicesManager().GetService( service_key )
service = HG.client_controller.services_manager.GetService( service_key )
service_type = service.GetServiceType()
@ -1176,7 +1176,7 @@ class Predicate( HydrusSerialisable.SerialisableBase ):
if current_or_pending == HC.CONTENT_STATUS_PENDING: base += u' pending to '
else: base += u' currently in '
service = HG.client_controller.GetServicesManager().GetService( service_key )
service = HG.client_controller.services_manager.GetService( service_key )
base += service.GetName()

4
include/ClientServices.py
View File

@ -1408,7 +1408,7 @@ class ServiceRepository( ServiceRestricted ):
if num_to_do > 0:
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
job_key = ClientThreading.JobKey( cancellable = True, stop_time = stop_time )
@ -1752,7 +1752,7 @@ class ServiceIPFS( ServiceRemote ):
url = api_base_url + 'add'
client_files_manager = HG.client_controller.GetClientFilesManager()
client_files_manager = HG.client_controller.client_files_manager
path = client_files_manager.GetFilePath( hash, mime )

4
include/ClientTags.py
View File

@ -78,7 +78,7 @@ def ImportFromHTA( parent, hta_path, tag_service_key, hashes ):
del hta
service = HG.client_controller.GetServicesManager().GetService( tag_service_key )
service = HG.client_controller.services_manager.GetService( tag_service_key )
service_type = service.GetServiceType()
@ -166,7 +166,7 @@ def ImportFromHTA( parent, hta_path, tag_service_key, hashes ):
text += os.linesep.join( HydrusData.ConvertUglyNamespacesToPrettyStrings( namespaces ) )
text += os.linesep * 2
file_service = HG.client_controller.GetServicesManager().GetService( file_service_key )
file_service = HG.client_controller.services_manager.GetService( file_service_key )
text += 'For '

20
include/HydrusConstants.py
View File

@ -49,7 +49,7 @@ options = {}
# Misc
NETWORK_VERSION = 18
SOFTWARE_VERSION = 261
SOFTWARE_VERSION = 262
UNSCALED_THUMBNAIL_DIMENSIONS = ( 200, 200 )
@ -418,7 +418,7 @@ UNDETERMINED_WM = 19
VIDEO_MKV = 20
VIDEO_WEBM = 21
APPLICATION_JSON = 22
VIDEO_APNG = 23
IMAGE_APNG = 23
UNDETERMINED_PNG = 24
VIDEO_MPEG = 25
VIDEO_MOV = 26
@ -429,16 +429,16 @@ TEXT_PLAIN = 30
APPLICATION_OCTET_STREAM = 100
APPLICATION_UNKNOWN = 101
ALLOWED_MIMES = ( IMAGE_JPEG, IMAGE_PNG, IMAGE_GIF, IMAGE_BMP, APPLICATION_FLASH, VIDEO_AVI, VIDEO_FLV, VIDEO_MOV, VIDEO_MP4, VIDEO_MKV, VIDEO_WEBM, VIDEO_MPEG, APPLICATION_PDF, AUDIO_MP3, AUDIO_OGG, AUDIO_FLAC, AUDIO_WMA, VIDEO_WMV, APPLICATION_HYDRUS_UPDATE_CONTENT, APPLICATION_HYDRUS_UPDATE_DEFINITIONS )
SEARCHABLE_MIMES = ( IMAGE_JPEG, IMAGE_PNG, IMAGE_GIF, APPLICATION_FLASH, VIDEO_AVI, VIDEO_FLV, VIDEO_MOV, VIDEO_MP4, VIDEO_MKV, VIDEO_WEBM, VIDEO_MPEG, APPLICATION_PDF, AUDIO_MP3, AUDIO_OGG, AUDIO_FLAC, AUDIO_WMA, VIDEO_WMV )
ALLOWED_MIMES = ( IMAGE_JPEG, IMAGE_PNG, IMAGE_APNG, IMAGE_GIF, IMAGE_BMP, APPLICATION_FLASH, VIDEO_AVI, VIDEO_FLV, VIDEO_MOV, VIDEO_MP4, VIDEO_MKV, VIDEO_WEBM, VIDEO_MPEG, APPLICATION_PDF, AUDIO_MP3, AUDIO_OGG, AUDIO_FLAC, AUDIO_WMA, VIDEO_WMV, APPLICATION_HYDRUS_UPDATE_CONTENT, APPLICATION_HYDRUS_UPDATE_DEFINITIONS )
SEARCHABLE_MIMES = ( IMAGE_JPEG, IMAGE_PNG, IMAGE_APNG, IMAGE_GIF, APPLICATION_FLASH, VIDEO_AVI, VIDEO_FLV, VIDEO_MOV, VIDEO_MP4, VIDEO_MKV, VIDEO_WEBM, VIDEO_MPEG, APPLICATION_PDF, AUDIO_MP3, AUDIO_OGG, AUDIO_FLAC, AUDIO_WMA, VIDEO_WMV )
IMAGES = ( IMAGE_JPEG, IMAGE_PNG, IMAGE_GIF, IMAGE_BMP )
IMAGES = ( IMAGE_JPEG, IMAGE_PNG, IMAGE_APNG, IMAGE_GIF, IMAGE_BMP )
AUDIO = ( AUDIO_MP3, AUDIO_OGG, AUDIO_FLAC, AUDIO_WMA )
VIDEO = ( VIDEO_AVI, VIDEO_FLV, VIDEO_MOV, VIDEO_MP4, VIDEO_WMV, VIDEO_MKV, VIDEO_WEBM, VIDEO_MPEG )
NATIVE_VIDEO = ( VIDEO_AVI, VIDEO_FLV, VIDEO_MOV, VIDEO_MP4, VIDEO_WMV, VIDEO_MKV, VIDEO_WEBM, VIDEO_MPEG )
NATIVE_VIDEO = ( IMAGE_APNG, VIDEO_AVI, VIDEO_FLV, VIDEO_MOV, VIDEO_MP4, VIDEO_WMV, VIDEO_MKV, VIDEO_WEBM, VIDEO_MPEG )
APPLICATIONS = ( APPLICATION_FLASH, APPLICATION_PDF, APPLICATION_ZIP )
@ -446,7 +446,7 @@ NOISY_MIMES = tuple( [ APPLICATION_FLASH ] + list( AUDIO ) + list( VIDEO ) )
ARCHIVES = ( APPLICATION_ZIP, APPLICATION_HYDRUS_ENCRYPTED_ZIP )
MIMES_WITH_THUMBNAILS = ( APPLICATION_FLASH, IMAGE_JPEG, IMAGE_PNG, IMAGE_GIF, IMAGE_BMP, VIDEO_AVI, VIDEO_FLV, VIDEO_MOV, VIDEO_MP4, VIDEO_WMV, VIDEO_MKV, VIDEO_WEBM, VIDEO_MPEG )
MIMES_WITH_THUMBNAILS = ( APPLICATION_FLASH, IMAGE_JPEG, IMAGE_PNG, IMAGE_APNG, IMAGE_GIF, IMAGE_BMP, VIDEO_AVI, VIDEO_FLV, VIDEO_MOV, VIDEO_MP4, VIDEO_WMV, VIDEO_MKV, VIDEO_WEBM, VIDEO_MPEG )
MIMES_WE_CAN_PHASH = ( IMAGE_JPEG, IMAGE_PNG )
@ -460,6 +460,7 @@ mime_enum_lookup[ 'image/jpeg' ] = IMAGE_JPEG
mime_enum_lookup[ 'image/jpg' ] = IMAGE_JPEG
mime_enum_lookup[ 'image/x-png' ] = IMAGE_PNG
mime_enum_lookup[ 'image/png' ] = IMAGE_PNG
mime_enum_lookup[ 'image/apng' ] = IMAGE_APNG
mime_enum_lookup[ 'image/gif' ] = IMAGE_GIF
mime_enum_lookup[ 'image/bmp' ] = IMAGE_BMP
mime_enum_lookup[ 'image' ] = IMAGES
@ -481,7 +482,6 @@ mime_enum_lookup[ 'audio/flac' ] = AUDIO_FLAC
mime_enum_lookup[ 'audio/x-ms-wma' ] = AUDIO_WMA
mime_enum_lookup[ 'text/html' ] = TEXT_HTML
mime_enum_lookup[ 'text/plain' ] = TEXT_PLAIN
mime_enum_lookup[ 'video/png' ] = VIDEO_APNG
mime_enum_lookup[ 'video/x-msvideo' ] = VIDEO_AVI
mime_enum_lookup[ 'video/x-flv' ] = VIDEO_FLV
mime_enum_lookup[ 'video/quicktime' ] = VIDEO_MOV
@ -498,6 +498,7 @@ mime_string_lookup = {}
mime_string_lookup[ APPLICATION_HYDRUS_CLIENT_COLLECTION ] = 'collection'
mime_string_lookup[ IMAGE_JPEG ] = 'image/jpg'
mime_string_lookup[ IMAGE_PNG ] = 'image/png'
mime_string_lookup[ IMAGE_APNG ] = 'image/apng'
mime_string_lookup[ IMAGE_GIF ] = 'image/gif'
mime_string_lookup[ IMAGE_BMP ] = 'image/bmp'
mime_string_lookup[ IMAGES ] = 'image'
@ -519,7 +520,6 @@ mime_string_lookup[ AUDIO_WMA ] = 'audio/x-ms-wma'
mime_string_lookup[ AUDIO ] = 'audio'
mime_string_lookup[ TEXT_HTML ] = 'text/html'
mime_string_lookup[ TEXT_PLAIN ] = 'text/plain'
mime_string_lookup[ VIDEO_APNG ] = 'video/png'
mime_string_lookup[ VIDEO_AVI ] = 'video/x-msvideo'
mime_string_lookup[ VIDEO_FLV ] = 'video/x-flv'
mime_string_lookup[ VIDEO_MOV ] = 'video/quicktime'
@ -537,6 +537,7 @@ mime_ext_lookup = {}
mime_ext_lookup[ APPLICATION_HYDRUS_CLIENT_COLLECTION ] = '.collection'
mime_ext_lookup[ IMAGE_JPEG ] = '.jpg'
mime_ext_lookup[ IMAGE_PNG ] = '.png'
mime_ext_lookup[ IMAGE_APNG ] = '.png'
mime_ext_lookup[ IMAGE_GIF ] = '.gif'
mime_ext_lookup[ IMAGE_BMP ] = '.bmp'
mime_ext_lookup[ IMAGE_ICON ] = '.ico'
@ -555,7 +556,6 @@ mime_ext_lookup[ AUDIO_FLAC ] = '.flac'
mime_ext_lookup[ AUDIO_WMA ] = '.wma'
mime_ext_lookup[ TEXT_HTML ] = '.html'
mime_ext_lookup[ TEXT_PLAIN ] = '.txt'
mime_ext_lookup[ VIDEO_APNG ] = '.png'
mime_ext_lookup[ VIDEO_AVI ] = '.avi'
mime_ext_lookup[ VIDEO_FLV ] = '.flv'
mime_ext_lookup[ VIDEO_MOV ] = '.mov'

156
include/HydrusFileHandling.py
View File

@ -70,60 +70,77 @@ def GenerateThumbnail( path, dimensions = HC.UNSCALED_THUMBNAIL_DIMENSIONS ):
mime = GetMime( path )
f = cStringIO.StringIO()
if mime in HC.IMAGES:
if mime in ( HC.IMAGE_JPEG, HC.IMAGE_PNG, HC.IMAGE_GIF ):
pil_image = HydrusImageHandling.GeneratePILImage( path )
SaveThumbnailToStream( pil_image, dimensions, f )
elif mime == HC.APPLICATION_FLASH:
( os_file_handle, temp_path ) = HydrusPaths.GetTempPath()
try:
HydrusFlashHandling.RenderPageToFile( path, temp_path, 1 )
pil_image = HydrusImageHandling.GeneratePILImage( temp_path )
SaveThumbnailToStream( pil_image, dimensions, f )
except:
flash_default_path = os.path.join( HC.STATIC_DIR, 'flash.png' )
pil_image = HydrusImageHandling.GeneratePILImage( flash_default_path )
SaveThumbnailToStream( pil_image, dimensions, f )
finally:
del pil_image
HydrusPaths.CleanUpTempPath( os_file_handle, temp_path )
thumbnail = GenerateThumbnailFromStaticImage( path, dimensions )
else:
( size, mime, width, height, duration, num_frames, num_words ) = GetFileInfo( path )
f = cStringIO.StringIO()
cropped_dimensions = HydrusImageHandling.GetThumbnailResolution( ( width, height ), dimensions )
renderer = HydrusVideoHandling.VideoRendererFFMPEG( path, mime, duration, num_frames, cropped_dimensions )
numpy_image = renderer.read_frame()
if numpy_image is None:
if mime == HC.APPLICATION_FLASH:
raise Exception( 'Could not create a thumbnail from that video!' )
( os_file_handle, temp_path ) = HydrusPaths.GetTempPath()
try:
HydrusFlashHandling.RenderPageToFile( path, temp_path, 1 )
pil_image = HydrusImageHandling.GeneratePILImage( temp_path )
SaveThumbnailToStream( pil_image, dimensions, f )
except:
flash_default_path = os.path.join( HC.STATIC_DIR, 'flash.png' )
pil_image = HydrusImageHandling.GeneratePILImage( flash_default_path )
SaveThumbnailToStream( pil_image, dimensions, f )
finally:
del pil_image
HydrusPaths.CleanUpTempPath( os_file_handle, temp_path )
else:
( size, mime, width, height, duration, num_frames, num_words ) = GetFileInfo( path )
cropped_dimensions = HydrusImageHandling.GetThumbnailResolution( ( width, height ), dimensions )
renderer = HydrusVideoHandling.VideoRendererFFMPEG( path, mime, duration, num_frames, cropped_dimensions )
numpy_image = renderer.read_frame()
if numpy_image is None:
raise Exception( 'Could not create a thumbnail from that video!' )
pil_image = HydrusImageHandling.GeneratePILImageFromNumpyImage( numpy_image )
SaveThumbnailToStream( pil_image, dimensions, f )
pil_image = HydrusImageHandling.GeneratePILImageFromNumpyImage( numpy_image )
f.seek( 0 )
SaveThumbnailToStream( pil_image, dimensions, f )
thumbnail = f.read()
f.close()
return thumbnail
def GenerateThumbnailFromStaticImage( path, dimensions = HC.UNSCALED_THUMBNAIL_DIMENSIONS ):
f = cStringIO.StringIO()
pil_image = HydrusImageHandling.GeneratePILImage( path )
SaveThumbnailToStream( pil_image, dimensions, f )
f.seek( 0 )
@ -174,7 +191,7 @@ def GetFileInfo( path ):
num_frames = None
num_words = None
if mime in HC.IMAGES:
if mime in ( HC.IMAGE_JPEG, HC.IMAGE_PNG, HC.IMAGE_GIF ):
( ( width, height ), duration, num_frames ) = HydrusImageHandling.GetImageProperties( path )
@ -182,15 +199,30 @@ def GetFileInfo( path ):
( ( width, height ), duration, num_frames ) = HydrusFlashHandling.GetFlashProperties( path )
elif mime in ( HC.VIDEO_AVI, HC.VIDEO_FLV, HC.VIDEO_WMV, HC.VIDEO_MOV, HC.VIDEO_MP4, HC.VIDEO_MKV, HC.VIDEO_WEBM, HC.VIDEO_MPEG ):
elif mime in ( HC.IMAGE_APNG, HC.VIDEO_AVI, HC.VIDEO_FLV, HC.VIDEO_WMV, HC.VIDEO_MOV, HC.VIDEO_MP4, HC.VIDEO_MKV, HC.VIDEO_WEBM, HC.VIDEO_MPEG ):
( ( width, height ), duration, num_frames ) = HydrusVideoHandling.GetFFMPEGVideoProperties( path )
elif mime == HC.APPLICATION_PDF: num_words = HydrusDocumentHandling.GetPDFNumWords( path )
elif mime == HC.AUDIO_MP3: duration = HydrusAudioHandling.GetMP3Duration( path )
elif mime == HC.AUDIO_OGG: duration = HydrusAudioHandling.GetOGGVorbisDuration( path )
elif mime == HC.AUDIO_FLAC: duration = HydrusAudioHandling.GetFLACDuration( path )
elif mime == HC.AUDIO_WMA: duration = HydrusAudioHandling.GetWMADuration( path )
elif mime == HC.APPLICATION_PDF:
num_words = HydrusDocumentHandling.GetPDFNumWords( path )
elif mime == HC.AUDIO_MP3:
duration = HydrusAudioHandling.GetMP3Duration( path )
elif mime == HC.AUDIO_OGG:
duration = HydrusAudioHandling.GetOGGVorbisDuration( path )
elif mime == HC.AUDIO_FLAC:
duration = HydrusAudioHandling.GetFLACDuration( path )
elif mime == HC.AUDIO_WMA:
duration = HydrusAudioHandling.GetWMADuration( path )
return ( size, mime, width, height, duration, num_frames, num_words )
@ -231,18 +263,14 @@ def GetMime( path ):
elif mime == HC.UNDETERMINED_PNG:
return HC.IMAGE_PNG
# atm (Feb 2016), ffmpeg doesn't report duration for apngs, so can't do this just yet.
#
#if HydrusVideoHandling.HasVideoStream( path ):
#
# return HC.VIDEO_APNG
#
#else:
#
# return HC.IMAGE_PNG
#
if HydrusVideoHandling.HasVideoStream( path ):
return HC.IMAGE_APNG
else:
return HC.IMAGE_PNG
else:
@ -253,7 +281,7 @@ def GetMime( path ):
try:
mime = HydrusVideoHandling.GetMimeFromFFMPEG( path )
mime = HydrusVideoHandling.GetMime( path )
if mime != HC.APPLICATION_UNKNOWN:

30
include/HydrusNetworking.py
View File

@ -105,14 +105,21 @@ class BandwidthRules( HydrusSerialisable.SerialisableBase ):
def CanContinue( self, bandwidth_tracker, threshold = 60 ):
def CanContinue( self, bandwidth_tracker, threshold = 15 ):
with self._lock:
for ( bandwidth_type, time_delta, max_allowed ) in self._rules:
# Do not stop ongoing just because starts are throttled
requests_rule = bandwidth_type == HC.BANDWIDTH_TYPE_REQUESTS
# Do not block an ongoing jpg download because the current month is 100.03% used
if time_delta is None or time_delta > threshold:
wait_is_too_long = time_delta is None or time_delta > threshold
ignore_rule = requests_rule or wait_is_too_long
if ignore_rule:
continue
@ -127,19 +134,21 @@ class BandwidthRules( HydrusSerialisable.SerialisableBase ):
def CanStart( self, bandwidth_tracker, threshold = 60 ):
def CanStart( self, bandwidth_tracker, threshold = 5 ):
with self._lock:
for ( bandwidth_type, time_delta, max_allowed ) in self._rules:
# Do not prohibit a new job from starting just because the current download speed is 210/200KB/s
if time_delta is not None and time_delta < threshold:
ignore_rule_for_starting = bandwidth_type == HC.BANDWIDTH_TYPE_DATA and time_delta is not None and time_delta <= threshold
if ignore_rule_for_starting:
continue
if bandwidth_tracker.GetUsage( bandwidth_type, time_delta ) > max_allowed:
if bandwidth_tracker.GetUsage( bandwidth_type, time_delta ) >= max_allowed:
return False
@ -266,6 +275,11 @@ class BandwidthTracker( HydrusSerialisable.SerialisableBase ):
self._seconds_requests = counters[ 9 ]
def _GetCurrentDateTime( self ):
return datetime.datetime.utcfromtimestamp( HydrusData.GetNow() )
def _GetMonthTime( self, dt ):
( year, month ) = ( dt.year, dt.month )
@ -281,7 +295,7 @@ class BandwidthTracker( HydrusSerialisable.SerialisableBase ):
if time_delta is None:
dt = datetime.datetime.utcnow()
dt = self._GetCurrentDateTime()
month_time = self._GetMonthTime( dt )
@ -473,7 +487,7 @@ class BandwidthTracker( HydrusSerialisable.SerialisableBase ):
with self._lock:
dt = datetime.datetime.utcnow()
dt = self._GetCurrentDateTime()
( month_time, day_time, hour_time, minute_time, second_time ) = self._GetTimes( dt )
@ -495,7 +509,7 @@ class BandwidthTracker( HydrusSerialisable.SerialisableBase ):
with self._lock:
dt = datetime.datetime.utcnow()
dt = self._GetCurrentDateTime()
( month_time, day_time, hour_time, minute_time, second_time ) = self._GetTimes( dt )

3
include/HydrusSerialisable.py
View File

@ -48,6 +48,9 @@ SERIALISABLE_TYPE_APPLICATION_COMMAND = 42
SERIALISABLE_TYPE_DUPLICATE_ACTION_OPTIONS = 43
SERIALISABLE_TYPE_TAG_CENSOR = 44
SERIALISABLE_TYPE_BANDWIDTH_MANAGER = 45
SERIALISABLE_TYPE_NETWORK_SESSION_MANAGER = 46
SERIALISABLE_TYPE_NETWORK_CONTEXT = 47
SERIALISABLE_TYPE_NETWORK_LOGIN_MANAGER = 48
SERIALISABLE_TYPES_TO_OBJECT_TYPES = {}

576
include/HydrusVideoHandling.py
View File

@ -27,6 +27,18 @@ if not os.path.exists( FFMPEG_PATH ):
FFMPEG_PATH = os.path.basename( FFMPEG_PATH )
def CheckFFMPEGError( lines ):
if "No such file or directory" in lines[-1]:
raise IOError( "File not found!" )
if 'Invalid data' in lines[-1]:
raise HydrusExceptions.MimeException( 'FFMPEG could not parse.' )
def GetFFMPEGVersion():
# open the file in a pipe, provoke an error, read output
@ -78,19 +90,158 @@ def GetFFMPEGVersion():
return 'unknown'
# bits of this were originally cribbed from moviepy
def GetFFMPEGInfoLines( path, count_frames_manually = False ):
# open the file in a pipe, provoke an error, read output
try:
path.encode( 'ascii' ) # throwing unicode at the console is a mess best left for Python 3
except UnicodeEncodeError:
( os_file_handle, temp_path ) = HydrusPaths.GetTempPath()
with open( path, 'rb' ) as source:
with open( temp_path, 'wb' ) as dest:
HydrusPaths.CopyFileLikeToFileLike( source, dest )
try:
return GetFFMPEGInfoLines( temp_path, count_frames_manually )
finally:
HydrusPaths.CleanUpTempPath( os_file_handle, temp_path )
cmd = [ FFMPEG_PATH, "-i", path ]
if count_frames_manually:
if HC.PLATFORM_WINDOWS:
cmd += [ "-f", "null", "NUL" ]
else:
cmd += [ "-f", "null", "/dev/null" ]
try:
proc = subprocess.Popen( cmd, bufsize = 10**5, stdout = subprocess.PIPE, stderr = subprocess.PIPE, startupinfo = HydrusData.GetHideTerminalSubprocessStartupInfo() )
except:
if not os.path.exists( FFMPEG_PATH ):
raise Exception( 'FFMPEG was not found!' )
else:
raise
raw_info = proc.stderr.read()
try:
info = raw_info.decode( 'utf8' )
except UnicodeDecodeError:
info = raw_info
proc.wait()
proc.communicate()
del proc
lines = info.splitlines()
CheckFFMPEGError( lines )
return lines
def GetFFMPEGVideoProperties( path, count_frames_manually = False ):
info = Hydrusffmpeg_parse_infos( path, count_frames_manually = count_frames_manually )
lines = GetFFMPEGInfoLines( path )
( w, h ) = info[ 'video_size' ]
if not ParseFFMPEGHasVideo( lines ):
raise HydrusExceptions.MimeException( 'File did not appear to have a video stream!' )
duration_in_s = info[ 'duration' ]
resolution = ParseFFMPEGVideoResolution( lines )
duration = int( duration_in_s * 1000 )
duration = ParseFFMPEGDuration( lines )
num_frames = info[ 'video_nframes' ]
if duration is None:
fps = ParseFFMPEGFPS( lines )
if fps is None:
raise HydrusExceptions.MimeException( 'Could not determine either the duration or fps!' )
if not count_frames_manually:
count_frames_manually = True
lines = GetFFMPEGInfoLines( path, count_frames_manually )
num_frames = ParseFFMPEGNumFramesManually( lines )
duration = num_frames / fps
else:
if not count_frames_manually:
fps = ParseFFMPEGFPS( lines )
it_was_accurate = fps is not None
if it_was_accurate:
num_frames = duration * fps
if num_frames != int( num_frames ): # we want whole numbers--anything else suggests start_offset is off or whatever
it_was_accurate = False
if not it_was_accurate:
count_frames_manually = True
lines = GetFFMPEGInfoLines( path, count_frames_manually )
if count_frames_manually:
num_frames = ParseFFMPEGNumFramesManually( lines )
return ( ( w, h ), duration, num_frames )
duration_in_ms = int( duration * 1000 )
return ( resolution, duration_in_ms, num_frames )
def GetMatroskaOrWebm( path ):
@ -143,46 +294,50 @@ def GetMatroskaOrWebMProperties( path ):
return ( ( width, height ), duration, num_frames )
def GetMimeFromFFMPEG( path ):
def GetMime( path ):
info = Hydrusffmpeg_parse_infos( path )
lines = GetFFMPEGInfoLines( path )
if 'mime_text' in info:
try:
mime_text = info[ 'mime_text' ]
mime_text = ParseFFMPEGMimeText( lines )
if 'matroska' in mime_text or 'webm' in mime_text:
except HydrusExceptions.MimeException:
return HC.APPLICATION_UNKNOWN
if 'matroska' in mime_text or 'webm' in mime_text:
# typically it is 'matroska,webm'
return GetMatroskaOrWebm( path )
elif mime_text in ( 'mpeg', 'mpegvideo', 'mpegts' ):
return HC.VIDEO_MPEG
elif mime_text == 'flac':
return HC.AUDIO_FLAC
elif mime_text == 'mp3':
return HC.AUDIO_MP3
elif mime_text == 'ogg':
return HC.AUDIO_OGG
elif mime_text == 'asf':
if ParseFFMPEGHasVideo( lines ):
# typically it is 'matroska,webm'
return HC.VIDEO_WMV
return GetMatroskaOrWebm( path )
else:
elif mime_text in ( 'mpeg', 'mpegvideo', 'mpegts' ):
return HC.VIDEO_MPEG
elif mime_text == 'flac':
return HC.AUDIO_FLAC
elif mime_text == 'mp3':
return HC.AUDIO_MP3
elif mime_text == 'ogg':
return HC.AUDIO_OGG
elif mime_text == 'asf':
if info[ 'video_found' ]:
return HC.VIDEO_WMV
else:
return HC.AUDIO_WMA
return HC.AUDIO_WMA
@ -190,122 +345,38 @@ def GetMimeFromFFMPEG( path ):
def HasVideoStream( path ):
try:
info = Hydrusffmpeg_parse_infos( path )
except IOError as e:
HydrusData.ShowException( 'Determining the mime for the file at ' + path + ' caused the following problem:' )
HydrusData.ShowException( e )
return False
lines = GetFFMPEGInfoLines( path )
return info[ 'video_found' ]
return ParseFFMPEGHasVideo( lines )
# this is cribbed from moviepy
def Hydrusffmpeg_parse_infos(filename, print_infos=False, count_frames_manually = False ):
"""Get file infos using ffmpeg.
Returns a dictionnary with the fields:
"video_found", "video_fps", "duration", "video_nframes",
"video_duration"
"audio_found", "audio_fps"
"video_duration" is slightly smaller than "duration" to avoid
fetching the uncomplete frames at the end, which raises an error.
"""
def ParseFFMPEGAudio( lines ):
# open the file in a pipe, provoke an error, read output
# this is from the old stuff--might be helpful later when we add audio
try:
filename.encode( 'ascii' ) # throwing unicode at the console is a mess best left for Python 3
except UnicodeEncodeError:
( os_file_handle, temp_path ) = HydrusPaths.GetTempPath()
with open( filename, 'rb' ) as source:
with open( temp_path, 'wb' ) as dest:
HydrusPaths.CopyFileLikeToFileLike( source, dest )
lines_audio = [l for l in lines if ' Audio: ' in l]
audio_found = lines_audio != []
if audio_found:
line = lines_audio[0]
try:
return Hydrusffmpeg_parse_infos( temp_path )
finally:
HydrusPaths.CleanUpTempPath( os_file_handle, temp_path )
match = re.search(" [0-9]* Hz", line)
audio_fps = int(line[match.start()+1:match.end()])
except:
audio_fps = 'unknown'
cmd = [ FFMPEG_PATH, "-i", filename ]
is_GIF = filename.endswith('.gif')
doing_manual_frame_count = is_GIF or count_frames_manually
if doing_manual_frame_count:
if HC.PLATFORM_WINDOWS: cmd += ["-f", "null", "NUL"]
else: cmd += ["-f", "null", "/dev/null"]
try:
proc = subprocess.Popen( cmd, bufsize=10**5, stdout=subprocess.PIPE, stderr=subprocess.PIPE, startupinfo = HydrusData.GetHideTerminalSubprocessStartupInfo() )
except:
if not os.path.exists( FFMPEG_PATH ):
raise Exception( 'FFMPEG was not found!' )
else:
raise
raw_infos = proc.stderr.read()
try:
infos = raw_infos.decode( 'utf8' )
except UnicodeDecodeError:
infos = raw_infos
proc.wait()
proc.communicate()
del proc
if print_infos:
# print the whole info text returned by FFMPEG
HydrusData.Print( infos )
lines = infos.splitlines()
if "No such file or directory" in lines[-1]:
raise IOError("%s not found ! Wrong path ?"%filename)
if 'Invalid data' in lines[-1]:
raise HydrusExceptions.MimeException( 'FFMPEG could not parse.' )
result = dict()
def ParseFFMPEGDuration( lines ):
# get duration (in seconds)
# Duration: 00:00:02.46, start: 0.033000, bitrate: 1069 kb/s
try:
keyword = ('frame=' if is_GIF else 'Duration: ')
line = [l for l in lines if keyword in l][0]
line = [ l for l in lines if 'Duration:' in l ][0]
if 'Duration: N/A' in line:
return None
if 'start:' in line:
@ -326,18 +397,85 @@ def Hydrusffmpeg_parse_infos(filename, print_infos=False, count_frames_manually
match = re.search("[0-9][0-9]:[0-9][0-9]:[0-9][0-9].[0-9][0-9]", line)
hms = map(float, line[match.start()+1:match.end()].split(':'))
if len(hms) == 1:
result['duration'] = hms[0]
elif len(hms) == 2:
result['duration'] = 60*hms[0]+hms[1]
elif len(hms) ==3:
result['duration'] = 3600*hms[0]+60*hms[1]+hms[2]
if len( hms ) == 1:
duration = hms[0]
elif len( hms ) == 2:
duration = 60 * hms[0] + hms[1]
elif len( hms ) ==3:
duration = 3600 * hms[0] + 60 * hms[1] + hms[2]
result[ 'duration' ] -= start_offset
duration -= start_offset
return duration
except:
raise IOError("Error reading duration in file %s,"%(filename)+
"Text parsed: %s"%infos)
raise HydrusExceptions.MimeException( 'Error reading duration!' )
def ParseFFMPEGFPS( lines ):
try:
line = ParseFFMPEGVideoLine( lines )
# get the frame rate
match = re.search("( [0-9]*.| )[0-9]* tbr", line)
if match is not None:
fps = line[match.start():match.end()].split(' ')[1]
tbr_fps_is_likely_garbage = match is None or fps.endswith( 'k' ) or float( fps ) > 60
if tbr_fps_is_likely_garbage:
match = re.search("( [0-9]*.| )[0-9]* fps", line)
if match is not None:
fps = line[match.start():match.end()].split(' ')[1]
fps_is_likely_garbage = match is None or fps.endswith( 'k' ) or float( fps ) > 60
if fps_is_likely_garbage:
return None
fps = float( fps )
return fps
except:
raise HydrusExceptions.MimeException( 'Error estimating framerate!' )
def ParseFFMPEGHasVideo( lines ):
try:
video_line = ParseFFMPEGVideoLine( lines )
except HydrusExceptions.MimeException:
return False
return True
def ParseFFMPEGMimeText( lines ):
try:
@ -349,116 +487,66 @@ def Hydrusffmpeg_parse_infos(filename, print_infos=False, count_frames_manually
mime_text = text.split( ', from' )[0]
result[ 'mime_text' ] = mime_text
return mime_text
except:
pass
raise HydrusExceptions.MimeException( 'Error reading mime!' )
if count_frames_manually:
def ParseFFMPEGNumFramesManually( lines ):
try:
frame_lines = [ l for l in lines if l.startswith( 'frame= ' ) ]
if len( frame_lines ) > 0:
l = frame_lines[-1] # there will be several of these, counting up as the file renders. we hence want the final one
while ' ' in l:
l = frame_lines[-1] # there will be several of these, counting up as the file renders. we hence want the final one
while ' ' in l:
l = l.replace( ' ', ' ' )
num_frames = int( l.split( ' ' )[1] )
result[ 'video_nframes' ] = num_frames
l = l.replace( ' ', ' ' )
num_frames = int( l.split( ' ' )[1] )
return num_frames
except:
raise HydrusExceptions.MimeException( 'Error counting number of frames!' )
def ParseFFMPEGVideoLine( lines ):
# get the output line that speaks about video
lines_video = [ l for l in lines if ' Video: ' in l and not ( ' Video: png' in l or ' Video: jpg' in l ) ] # mp3 says it has a 'png' video stream
result['video_found'] = ( lines_video != [] )
if result['video_found']:
if len( lines_video ) == 0:
raise HydrusExceptions.MimeException( 'Could not find video information!' )
line = lines_video[0]
return line
def ParseFFMPEGVideoResolution( lines ):
try:
line = ParseFFMPEGVideoLine( lines )
line = lines_video[0]
# get the size, of the form 460x320 (w x h)
match = re.search(" [0-9]*x[0-9]*(,| )", line)
s = list(map(int, line[match.start():match.end()-1].split('x')))
result['video_size'] = s
have_to_fetch_manually = False
resolution = list(map(int, line[match.start():match.end()-1].split('x')))
if 'video_nframes' in result:
result[ 'video_fps' ] = result[ 'video_nframes' ] / result[ 'duration' ]
else:
# get the frame rate
match = re.search("( [0-9]*.| )[0-9]* tbr", line)
if match is not None:
fps = line[match.start():match.end()].split(' ')[1]
if match is None or fps.endswith( 'k' ):
match = re.search("( [0-9]*.| )[0-9]* fps", line)
if match is not None:
fps = line[match.start():match.end()].split(' ')[1]
if match is None or fps.endswith( 'k' ) or float( fps ) > 60:
if not doing_manual_frame_count:
return Hydrusffmpeg_parse_infos( filename, count_frames_manually = True )
else:
raise Exception( 'Could not determine framerate!' )
result['video_fps'] = float( fps )
num_frames = result['duration'] * result['video_fps']
if num_frames != int( num_frames ):
return Hydrusffmpeg_parse_infos( filename, count_frames_manually = True )
result['video_nframes'] = int( num_frames )
return resolution
except:
raise HydrusExceptions.MimeException( 'Error counting number of frames!' )
result['video_duration'] = result['duration']
# We could have also recomputed the duration from the number
# of frames, as follows:
# >>> result['video_duration'] = result['video_nframes'] / result['video_fps']
lines_audio = [l for l in lines if ' Audio: ' in l]
result['audio_found'] = lines_audio != []
if result['audio_found']:
line = lines_audio[0]
try:
match = re.search(" [0-9]* Hz", line)
result['audio_fps'] = int(line[match.start()+1:match.end()])
except:
result['audio_fps'] = 'unknown'
return result
# This was built from moviepy's FFMPEG_VideoReader
class VideoRendererFFMPEG( object ):
@ -518,7 +606,7 @@ class VideoRendererFFMPEG( object ):
self.close()
if self._mime == HC.IMAGE_GIF:
if self._mime in ( HC.IMAGE_APNG, HC.IMAGE_GIF ):
ss = 0
self.pos = 0

4
include/TestClientDownloading.py
View File

@ -91,7 +91,7 @@ class TestDownloaders( unittest.TestCase ):
with open( os.path.join( HC.STATIC_DIR, 'testing', 'pixiv_gallery.html' ) ) as f: pixiv_gallery = f.read()
with open( os.path.join( HC.STATIC_DIR, 'testing', 'pixiv_image_page.html' ) ) as f: pixiv_page = f.read()
HG.test_controller.GetHTTP().SetResponse( HC.GET, 'http://www.pixiv.net/search.php?word=naruto&s_mode=s_tag_full&order=date_d&p=1', pixiv_gallery )
HG.test_controller.GetHTTP().SetResponse( HC.GET, 'https://www.pixiv.net/search.php?word=naruto&s_mode=s_tag_full&order=date_d&p=1', pixiv_gallery )
HG.test_controller.GetHTTP().SetResponse( HC.GET, 'http://www.pixiv.net/member_illust.php?mode=medium&illust_id=50926312', pixiv_page )
HG.test_controller.GetHTTP().SetResponse( HC.GET, 'http://i3.pixiv.net/img-original/img/2014/01/25/19/21/56/41171994_p0.jpg', 'image file' )
@ -106,7 +106,7 @@ class TestDownloaders( unittest.TestCase ):
# a manga one, http://www.pixiv.net/member_illust.php?mode=medium&illust_id=51078392, is currently filtered
expected_gallery_urls = [ u'http://www.pixiv.net/member_illust.php?mode=medium&illust_id=50926312', 'a bunch of others' ]
expected_gallery_urls = [ u'https://www.pixiv.net/member_illust.php?mode=medium&illust_id=50926312', 'a bunch of others' ]
self.assertEqual( page_of_urls[0], expected_gallery_urls[0] )

454
include/TestClientNetworking.py Normal file
View File

@ -0,0 +1,454 @@
import ClientConstants as CC
import ClientNetworking
import collections
import HydrusConstants as HC
import HydrusData
import HydrusNetworking
import os
import TestConstants
import threading
import time
import unittest
import HydrusGlobals as HG
from httmock import all_requests, urlmatch, HTTMock
from mock import patch
# some gumpf
GOOD_RESPONSE = ''.join( chr( i ) for i in range( 256 ) )
# 256KB of gumpf
LONG_GOOD_RESPONSE = GOOD_RESPONSE * 4 * 256
@all_requests
def catch_all( url, request ):
raise Exception( 'An unexpected request for ' + url + ' came through in testing.' )
MOCK_DOMAIN = 'wew.lad'
MOCK_SUBDOMAIN = 'top.wew.lad'
MOCK_URL = 'https://wew.lad/folder/request&key1=value1&key2=value2'
MOCK_SUBURL = 'https://top.wew.lad/folder2/request&key1=value1&key2=value2'
@urlmatch( netloc = 'wew.lad' )
def catch_wew_ok( url, request ):
return GOOD_RESPONSE
@urlmatch( netloc = '123.45.67.89:45871' )
def catch_hydrus_ok( url, request ):
return GOOD_RESPONSE
class TestBandwidthManager( unittest.TestCase ):
def test_can_start( self ):
EMPTY_RULES = HydrusNetworking.BandwidthRules()
PERMISSIVE_DATA_RULES = HydrusNetworking.BandwidthRules()
PERMISSIVE_DATA_RULES.AddRule( HC.BANDWIDTH_TYPE_DATA, None, 1048576 )
PERMISSIVE_REQUEST_RULES = HydrusNetworking.BandwidthRules()
PERMISSIVE_REQUEST_RULES.AddRule( HC.BANDWIDTH_TYPE_REQUESTS, None, 10000 )
RESTRICTIVE_DATA_RULES = HydrusNetworking.BandwidthRules()
RESTRICTIVE_DATA_RULES.AddRule( HC.BANDWIDTH_TYPE_DATA, None, 10 )
RESTRICTIVE_REQUEST_RULES = HydrusNetworking.BandwidthRules()
RESTRICTIVE_REQUEST_RULES.AddRule( HC.BANDWIDTH_TYPE_REQUESTS, None, 1 )
DOMAIN_NETWORK_CONTEXT = ClientNetworking.NetworkContext( CC.NETWORK_CONTEXT_DOMAIN, MOCK_DOMAIN )
SUBDOMAIN_NETWORK_CONTEXT = ClientNetworking.NetworkContext( CC.NETWORK_CONTEXT_DOMAIN, MOCK_SUBDOMAIN )
GLOBAL_NETWORK_CONTEXTS = [ ClientNetworking.GLOBAL_NETWORK_CONTEXT ]
DOMAIN_NETWORK_CONTEXTS = [ ClientNetworking.GLOBAL_NETWORK_CONTEXT, DOMAIN_NETWORK_CONTEXT ]
SUBDOMAIN_NETWORK_CONTEXTS = [ ClientNetworking.GLOBAL_NETWORK_CONTEXT, DOMAIN_NETWORK_CONTEXT, SUBDOMAIN_NETWORK_CONTEXT ]
#
fast_forward = HydrusData.GetNow() + 3600
with patch.object( HydrusData, 'GetNow', return_value = fast_forward ):
bm = ClientNetworking.NetworkBandwidthManager()
self.assertTrue( bm.CanStart( GLOBAL_NETWORK_CONTEXTS ) )
self.assertTrue( bm.CanStart( DOMAIN_NETWORK_CONTEXTS ) )
self.assertTrue( bm.CanStart( SUBDOMAIN_NETWORK_CONTEXTS ) )
#
bm.ReportRequestUsed( DOMAIN_NETWORK_CONTEXTS )
bm.ReportDataUsed( DOMAIN_NETWORK_CONTEXTS, 50 )
bm.ReportRequestUsed( SUBDOMAIN_NETWORK_CONTEXTS )
bm.ReportDataUsed( SUBDOMAIN_NETWORK_CONTEXTS, 25 )
self.assertTrue( bm.CanStart( GLOBAL_NETWORK_CONTEXTS ) )
self.assertTrue( bm.CanStart( DOMAIN_NETWORK_CONTEXTS ) )
self.assertTrue( bm.CanStart( SUBDOMAIN_NETWORK_CONTEXTS ) )
#
bm.SetRules( None, EMPTY_RULES )
bm.SetRules( MOCK_DOMAIN, EMPTY_RULES )
bm.SetRules( MOCK_SUBDOMAIN, EMPTY_RULES )
self.assertTrue( bm.CanStart( GLOBAL_NETWORK_CONTEXTS ) )
self.assertTrue( bm.CanStart( DOMAIN_NETWORK_CONTEXTS ) )
self.assertTrue( bm.CanStart( SUBDOMAIN_NETWORK_CONTEXTS ) )
bm.SetRules( None, PERMISSIVE_DATA_RULES )
bm.SetRules( MOCK_DOMAIN, PERMISSIVE_DATA_RULES )
bm.SetRules( MOCK_SUBDOMAIN, PERMISSIVE_DATA_RULES )
self.assertTrue( bm.CanStart( GLOBAL_NETWORK_CONTEXTS ) )
self.assertTrue( bm.CanStart( DOMAIN_NETWORK_CONTEXTS ) )
self.assertTrue( bm.CanStart( SUBDOMAIN_NETWORK_CONTEXTS ) )
bm.SetRules( None, PERMISSIVE_REQUEST_RULES )
bm.SetRules( MOCK_DOMAIN, PERMISSIVE_REQUEST_RULES )
bm.SetRules( MOCK_SUBDOMAIN, PERMISSIVE_REQUEST_RULES )
self.assertTrue( bm.CanStart( GLOBAL_NETWORK_CONTEXTS ) )
self.assertTrue( bm.CanStart( DOMAIN_NETWORK_CONTEXTS ) )
self.assertTrue( bm.CanStart( SUBDOMAIN_NETWORK_CONTEXTS ) )
#
bm.SetRules( MOCK_SUBDOMAIN, RESTRICTIVE_DATA_RULES )
self.assertTrue( bm.CanStart( GLOBAL_NETWORK_CONTEXTS ) )
self.assertTrue( bm.CanStart( DOMAIN_NETWORK_CONTEXTS ) )
self.assertFalse( bm.CanStart( SUBDOMAIN_NETWORK_CONTEXTS ) )
bm.SetRules( MOCK_SUBDOMAIN, RESTRICTIVE_REQUEST_RULES )
self.assertTrue( bm.CanStart( GLOBAL_NETWORK_CONTEXTS ) )
self.assertTrue( bm.CanStart( DOMAIN_NETWORK_CONTEXTS ) )
self.assertFalse( bm.CanStart( SUBDOMAIN_NETWORK_CONTEXTS ) )
bm.SetRules( MOCK_SUBDOMAIN, PERMISSIVE_REQUEST_RULES )
self.assertTrue( bm.CanStart( GLOBAL_NETWORK_CONTEXTS ) )
self.assertTrue( bm.CanStart( DOMAIN_NETWORK_CONTEXTS ) )
self.assertTrue( bm.CanStart( SUBDOMAIN_NETWORK_CONTEXTS ) )
#
bm.SetRules( MOCK_DOMAIN, RESTRICTIVE_DATA_RULES )
self.assertTrue( bm.CanStart( GLOBAL_NETWORK_CONTEXTS ) )
self.assertFalse( bm.CanStart( DOMAIN_NETWORK_CONTEXTS ) )
self.assertFalse( bm.CanStart( SUBDOMAIN_NETWORK_CONTEXTS ) )
bm.SetRules( MOCK_DOMAIN, RESTRICTIVE_REQUEST_RULES )
self.assertTrue( bm.CanStart( GLOBAL_NETWORK_CONTEXTS ) )
self.assertFalse( bm.CanStart( DOMAIN_NETWORK_CONTEXTS ) )
self.assertFalse( bm.CanStart( SUBDOMAIN_NETWORK_CONTEXTS ) )
bm.SetRules( MOCK_DOMAIN, PERMISSIVE_REQUEST_RULES )
self.assertTrue( bm.CanStart( GLOBAL_NETWORK_CONTEXTS ) )
self.assertTrue( bm.CanStart( DOMAIN_NETWORK_CONTEXTS ) )
self.assertTrue( bm.CanStart( MOCK_SUBDOMAIN ) )
#
bm.SetRules( None, RESTRICTIVE_DATA_RULES )
self.assertFalse( bm.CanStart( GLOBAL_NETWORK_CONTEXTS ) )
self.assertFalse( bm.CanStart( DOMAIN_NETWORK_CONTEXTS ) )
self.assertFalse( bm.CanStart( SUBDOMAIN_NETWORK_CONTEXTS ) )
bm.SetRules( None, RESTRICTIVE_REQUEST_RULES )
self.assertFalse( bm.CanStart( GLOBAL_NETWORK_CONTEXTS ) )
self.assertFalse( bm.CanStart( DOMAIN_NETWORK_CONTEXTS ) )
self.assertFalse( bm.CanStart( SUBDOMAIN_NETWORK_CONTEXTS ) )
bm.SetRules( None, PERMISSIVE_REQUEST_RULES )
self.assertTrue( bm.CanStart( GLOBAL_NETWORK_CONTEXTS ) )
self.assertTrue( bm.CanStart( DOMAIN_NETWORK_CONTEXTS ) )
self.assertTrue( bm.CanStart( MOCK_SUBDOMAIN ) )
# add some rules for all
bm.SetRules( None, RESTRICTIVE_DATA_RULES )
bm.SetRules( MOCK_DOMAIN, RESTRICTIVE_REQUEST_RULES )
bm.SetRules( MOCK_DOMAIN, EMPTY_RULES )
self.assertFalse( bm.CanStart( GLOBAL_NETWORK_CONTEXTS ) )
self.assertFalse( bm.CanStart( DOMAIN_NETWORK_CONTEXTS ) )
self.assertFalse( bm.CanStart( SUBDOMAIN_NETWORK_CONTEXTS ) )
def test_can_continue( self ):
pass
class TestNetworkingEngine( unittest.TestCase ):
def test_engine_shutdown_app( self ):
mock_controller = TestConstants.MockController()
bandwidth_manager = ClientNetworking.NetworkBandwidthManager()
session_manager = ClientNetworking.NetworkSessionManager()
login_manager = ClientNetworking.NetworkLoginManager()
engine = ClientNetworking.NetworkEngine( mock_controller, bandwidth_manager, session_manager, login_manager )
self.assertFalse( engine.IsRunning() )
self.assertFalse( engine.IsShutdown() )
engine.Start()
time.sleep( 0.1 )
self.assertTrue( engine.IsRunning() )
self.assertFalse( engine.IsShutdown() )
mock_controller.model_is_shutdown = True
engine._new_work_to_do.set()
time.sleep( 0.1 )
self.assertFalse( engine.IsRunning() )
self.assertTrue( engine.IsShutdown() )
def test_engine_shutdown_manual( self ):
mock_controller = TestConstants.MockController()
bandwidth_manager = ClientNetworking.NetworkBandwidthManager()
session_manager = ClientNetworking.NetworkSessionManager()
login_manager = ClientNetworking.NetworkLoginManager()
engine = ClientNetworking.NetworkEngine( mock_controller, bandwidth_manager, session_manager, login_manager )
self.assertFalse( engine.IsRunning() )
self.assertFalse( engine.IsShutdown() )
engine.Start()
time.sleep( 0.1 )
self.assertTrue( engine.IsRunning() )
self.assertFalse( engine.IsShutdown() )
engine.Shutdown()
time.sleep( 0.1 )
self.assertFalse( engine.IsRunning() )
self.assertTrue( engine.IsShutdown() )
class TestNetworkingJob( unittest.TestCase ):
def _GetJob( self ):
job = ClientNetworking.NetworkJob( 'GET', MOCK_URL )
mock_controller = TestConstants.MockController()
bandwidth_manager = ClientNetworking.NetworkBandwidthManager()
session_manager = ClientNetworking.NetworkSessionManager()
login_manager = ClientNetworking.NetworkLoginManager()
engine = ClientNetworking.NetworkEngine( mock_controller, bandwidth_manager, session_manager, login_manager )
job.engine = engine
return job
def test_cancelled_manually( self ):
job = self._GetJob()
self.assertFalse( job.IsCancelled() )
self.assertFalse( job.IsDone() )
job.Cancel()
self.assertTrue( job.IsCancelled() )
self.assertTrue( job.IsDone() )
def test_cancelled_app_shutdown( self ):
job = self._GetJob()
self.assertFalse( job.IsCancelled() )
self.assertFalse( job.IsDone() )
job.engine.controller.model_is_shutdown = True
self.assertTrue( job.IsCancelled() )
self.assertTrue( job.IsDone() )
def test_sleep( self ):
job = self._GetJob()
self.assertFalse( job.IsAsleep() )
job.Sleep( 3 )
self.assertTrue( job.IsAsleep() )
five_secs_from_now = HydrusData.GetNow() + 5
with patch.object( HydrusData, 'GetNow', return_value = five_secs_from_now ):
self.assertFalse( job.IsAsleep() )
class TestNetworkingJobWeb( unittest.TestCase ):
def _GetJob( self ):
job = ClientNetworking.NetworkJob( 'GET', MOCK_URL )
controller = TestConstants.MockController()
job.controller = controller
return job
def test_bandwidth_ok( self ):
# test bandwidth override
# test bandwidth ok
# test it not ok
# repeat for the login one
pass
def test_done_ok( self ):
return # need to flush out session, bandwidth, login code
with HTTMock( catch_all ):
with HTTMock( catch_wew_ok ):
job = self._GetJob()
job.Start()
self.assertFalse( job.HasError() )
self.assertEqual( job.GetContent(), GOOD_RESPONSE )
def test_error( self ):
job = self._GetJob()
# do a requests job that cancels
# haserror
# geterrorexception
# geterrortext
def test_generate_login_process( self ):
# test the system works as expected
pass
def test_needs_login( self ):
# test for both normal and login
pass
class TestNetworkingJobHydrus( unittest.TestCase ):
def _GetJob( self ):
job = ClientNetworking.NetworkJob( 'GET', 'https://123.45.67.89:45871/muh_hydrus_command' )
controller = TestConstants.MockController()
job.controller = controller
return job
def test_bandwidth_ok( self ):
# test bandwidth override
# test bandwidth ok
# test it not ok
# repeat for the login one
pass
def test_done_ok( self ):
return # need to flush out session, bandwidth, login code
with HTTMock( catch_all ):
with HTTMock( catch_hydrus_ok ):
job = self._GetJob()
job.Start()
self.assertFalse( job.HasError() )
self.assertEqual( job.GetContent(), GOOD_RESPONSE )
def test_error( self ):
job = self._GetJob()
# do a requests job that cancels
# haserror
# geterrorexception
# geterrortext
def test_generate_login_process( self ):
# test the system works as expected
pass
def test_needs_login( self ):
# test for both normal and login
pass

19
include/TestConstants.py
View File

@ -1,12 +1,14 @@
import collections
import ClientConstants as CC
import HydrusConstants as HC
import HydrusGlobals as HG
import HydrusTags
import os
import random
import threading
import weakref
import HydrusData
import HydrusThreading
import wx
DB_DIR = None
@ -27,6 +29,23 @@ def ConvertServiceKeysToContentUpdatesToComparable( service_keys_to_content_upda
return comparable_dict
class MockController( object ):
def __init__( self ):
self.model_is_shutdown = False
def CallToThread( self, callable, *args, **kwargs ):
return HG.test_controller.CallToThread( callable, *args, **kwargs )
def ModelIsShutdown( self ):
return self.model_is_shutdown
class FakeHTTPConnectionManager():
def __init__( self ):

1
include/TestDB.py
View File

@ -685,6 +685,7 @@ class TestClientDB( unittest.TestCase ):
test_files.append( ( 'muh_webm.webm', '55b6ce9d067326bf4b2fbe66b8f51f366bc6e5f776ba691b0351364383c43fcb', 84069, HC.VIDEO_WEBM, 640, 360, 4010, 120, None ) )
test_files.append( ( 'muh_jpg.jpg', '5d884d84813beeebd59a35e474fa3e4742d0f2b6679faa7609b245ddbbd05444', 42296, HC.IMAGE_JPEG, 392, 498, None, None, None ) )
test_files.append( ( 'muh_png.png', 'cdc67d3b377e6e1397ffa55edc5b50f6bdf4482c7a6102c6f27fa351429d6f49', 31452, HC.IMAGE_PNG, 191, 196, None, None, None ) )
test_files.append( ( 'muh_apng.png', '9e7b8b5abc7cb11da32db05671ce926a2a2b701415d1b2cb77a28deea51010c3', 616956, HC.IMAGE_APNG, 500, 500, 1880, 47, None ) )
test_files.append( ( 'muh_gif.gif', '00dd9e9611ebc929bfc78fde99a0c92800bbb09b9d18e0946cea94c099b211c2', 15660, HC.IMAGE_GIF, 329, 302, 600, 5, None ) )
for ( filename, hex_hash, size, mime, width, height, duration, num_frames, num_words ) in test_files:

584
include/TestHydrusNetworking.py
View File

@ -1,106 +1,524 @@
import collections
import HydrusConstants as HC
import os
import random
import TestConstants
import time
import unittest
import HydrusData
import HydrusGlobals as HG
import HydrusNetworking
from mock import patch
class TestBandwidthManagement( unittest.TestCase ):
now = HydrusData.GetNow()
now_10 = now + 10
now_20 = now + 20
with patch.object( HydrusData, 'GetNow', return_value = now ):
HIGH_USAGE = HydrusNetworking.BandwidthTracker()
for i in range( 100 ):
HIGH_USAGE.ReportRequestUsed()
HIGH_USAGE.ReportDataUsed( random.randint( 512, 1024 ) )
LOW_USAGE = HydrusNetworking.BandwidthTracker()
LOW_USAGE.ReportRequestUsed()
LOW_USAGE.ReportDataUsed( 1024 )
ZERO_USAGE = HydrusNetworking.BandwidthTracker()
class TestBandwidthRules( unittest.TestCase ):
def test_no_rules( self ):
rules = HydrusNetworking.BandwidthRules()
with patch.object( HydrusData, 'GetNow', return_value = now ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertTrue( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
def test_per_sec( self ):
# at short time deltas, we can always start based on data alone
rules = HydrusNetworking.BandwidthRules()
rules.AddRule( HC.BANDWIDTH_TYPE_DATA, 1, 10240 )
with patch.object( HydrusData, 'GetNow', return_value = now ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertTrue( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertFalse( rules.CanContinue( HIGH_USAGE ) )
with patch.object( HydrusData, 'GetNow', return_value = now_10 ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertTrue( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
with patch.object( HydrusData, 'GetNow', return_value = now_20 ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertTrue( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
#
rules = HydrusNetworking.BandwidthRules()
rules.AddRule( HC.BANDWIDTH_TYPE_REQUESTS, 1, 1 )
with patch.object( HydrusData, 'GetNow', return_value = now ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertFalse( rules.CanStart( LOW_USAGE ) )
self.assertFalse( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
with patch.object( HydrusData, 'GetNow', return_value = now_10 ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertTrue( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
with patch.object( HydrusData, 'GetNow', return_value = now_20 ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertTrue( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
#
rules = HydrusNetworking.BandwidthRules()
rules.AddRule( HC.BANDWIDTH_TYPE_DATA, 1, 10240 )
rules.AddRule( HC.BANDWIDTH_TYPE_REQUESTS, 1, 1 )
with patch.object( HydrusData, 'GetNow', return_value = now ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertFalse( rules.CanStart( LOW_USAGE ) )
self.assertFalse( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertFalse( rules.CanContinue( HIGH_USAGE ) )
with patch.object( HydrusData, 'GetNow', return_value = now_10 ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertTrue( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
with patch.object( HydrusData, 'GetNow', return_value = now_20 ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertTrue( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
def test_per_min( self ):
# cutoff is 15s for continue
rules = HydrusNetworking.BandwidthRules()
rules.AddRule( HC.BANDWIDTH_TYPE_DATA, 60, 10240 )
with patch.object( HydrusData, 'GetNow', return_value = now ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertFalse( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
with patch.object( HydrusData, 'GetNow', return_value = now_10 ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertFalse( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
with patch.object( HydrusData, 'GetNow', return_value = now_20 ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertFalse( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
#
rules = HydrusNetworking.BandwidthRules()
rules.AddRule( HC.BANDWIDTH_TYPE_REQUESTS, 60, 10 )
with patch.object( HydrusData, 'GetNow', return_value = now ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertFalse( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
with patch.object( HydrusData, 'GetNow', return_value = now_10 ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertFalse( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
with patch.object( HydrusData, 'GetNow', return_value = now_20 ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertFalse( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
#
rules = HydrusNetworking.BandwidthRules()
rules.AddRule( HC.BANDWIDTH_TYPE_DATA, 60, 10240 )
rules.AddRule( HC.BANDWIDTH_TYPE_REQUESTS, 60, 10 )
with patch.object( HydrusData, 'GetNow', return_value = now ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertFalse( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
with patch.object( HydrusData, 'GetNow', return_value = now_10 ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertFalse( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
with patch.object( HydrusData, 'GetNow', return_value = now_20 ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertFalse( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
def test_per_month( self ):
rules = HydrusNetworking.BandwidthRules()
rules.AddRule( HC.BANDWIDTH_TYPE_DATA, None, 10240 )
with patch.object( HydrusData, 'GetNow', return_value = now ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertFalse( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
with patch.object( HydrusData, 'GetNow', return_value = now_10 ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertFalse( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
with patch.object( HydrusData, 'GetNow', return_value = now_20 ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertFalse( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
#
rules = HydrusNetworking.BandwidthRules()
rules.AddRule( HC.BANDWIDTH_TYPE_REQUESTS, None, 10 )
with patch.object( HydrusData, 'GetNow', return_value = now ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertFalse( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
with patch.object( HydrusData, 'GetNow', return_value = now_10 ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertFalse( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
with patch.object( HydrusData, 'GetNow', return_value = now_20 ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertFalse( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
#
rules = HydrusNetworking.BandwidthRules()
rules.AddRule( HC.BANDWIDTH_TYPE_DATA, None, 10240 )
rules.AddRule( HC.BANDWIDTH_TYPE_REQUESTS, None, 10 )
with patch.object( HydrusData, 'GetNow', return_value = now ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertFalse( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
with patch.object( HydrusData, 'GetNow', return_value = now_10 ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertFalse( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
with patch.object( HydrusData, 'GetNow', return_value = now_20 ):
self.assertTrue( rules.CanStart( ZERO_USAGE ) )
self.assertTrue( rules.CanStart( LOW_USAGE ) )
self.assertFalse( rules.CanStart( HIGH_USAGE ) )
self.assertTrue( rules.CanContinue( ZERO_USAGE ) )
self.assertTrue( rules.CanContinue( LOW_USAGE ) )
self.assertTrue( rules.CanContinue( HIGH_USAGE ) )
class TestBandwidthTracker( unittest.TestCase ):
def test_bandwidth_tracker( self ):
bandwidth_tracker = HydrusNetworking.BandwidthTracker()
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 0 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 0 ), 0 )
self.assertEqual( bandwidth_tracker.GetCurrentMonthSummary(), 'used 0.0B in 0 requests this month' )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 1 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 1 ), 0 )
now = HydrusData.GetNow()
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 2 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 2 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 6 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 6 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 3600 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 3600 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, None ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, None ), 0 )
self.assertAlmostEquals
#
bandwidth_tracker.ReportDataUsed( 1024 )
bandwidth_tracker.ReportRequestUsed()
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 0 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 0 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 1 ), 170 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 1 ), 1 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 2 ), 85 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 2 ), 1 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 6 ), 1024 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 6 ), 1 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 3600 ), 1024 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 3600 ), 1 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, None ), 1024 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, None ), 1 )
with patch.object( HydrusData, 'GetNow', return_value = now ):
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 0 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 0 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 1 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 1 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 2 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 2 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 6 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 6 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 3600 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 3600 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, None ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, None ), 0 )
#
bandwidth_tracker.ReportDataUsed( 1024 )
bandwidth_tracker.ReportRequestUsed()
self.assertEqual( bandwidth_tracker.GetCurrentMonthSummary(), 'used 1024B in 1 requests this month' )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 0 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 0 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 1 ), 170 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 1 ), 1 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 2 ), 85 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 2 ), 1 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 6 ), 1024 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 6 ), 1 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 3600 ), 1024 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 3600 ), 1 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, None ), 1024 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, None ), 1 )
#
time.sleep( 5 )
five_secs_from_now = now + 5
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 0 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 0 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 1 ), 42 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 1 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 2 ), 85 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 2 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 6 ), 1024 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 6 ), 1 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 3600 ), 1024 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 3600 ), 1 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, None ), 1024 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, None ), 1 )
#
bandwidth_tracker.ReportDataUsed( 32 )
bandwidth_tracker.ReportRequestUsed()
bandwidth_tracker.ReportDataUsed( 32 )
bandwidth_tracker.ReportRequestUsed()
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 0 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 0 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 1 ), 53 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 1 ), 2 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 2 ), 90 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 2 ), 2 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 6 ), 1088 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 6 ), 3 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 3600 ), 1088 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 3600 ), 3 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, None ), 1088 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, None ), 3 )
with patch.object( HydrusData, 'GetNow', return_value = five_secs_from_now ):
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 0 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 0 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 1 ), 42 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 1 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 2 ), 85 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 2 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 6 ), 1024 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 6 ), 1 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 3600 ), 1024 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 3600 ), 1 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, None ), 1024 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, None ), 1 )
#
bandwidth_tracker.ReportDataUsed( 32 )
bandwidth_tracker.ReportRequestUsed()
bandwidth_tracker.ReportDataUsed( 32 )
bandwidth_tracker.ReportRequestUsed()
self.assertEqual( bandwidth_tracker.GetCurrentMonthSummary(), 'used 1.1KB in 3 requests this month' )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 0 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 0 ), 0 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 1 ), 53 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 1 ), 2 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 2 ), 90 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 2 ), 2 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 6 ), 1088 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 6 ), 3 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, 3600 ), 1088 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, 3600 ), 3 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_DATA, None ), 1088 )
self.assertEqual( bandwidth_tracker.GetUsage( HC.BANDWIDTH_TYPE_REQUESTS, None ), 3 )

2
include/TestHydrusServer.py
View File

@ -62,7 +62,7 @@ class TestServer( unittest.TestCase ):
cls._local_booru = ClientServices.GenerateService( HydrusData.GenerateKey(), HC.LOCAL_BOORU, 'local booru' )
services_manager = HG.test_controller.GetServicesManager()
services_manager = HG.test_controller.services_manager
services_manager._keys_to_services[ cls._clientside_file_service.GetServiceKey() ] = cls._clientside_file_service
services_manager._keys_to_services[ cls._clientside_tag_service.GetServiceKey() ] = cls._clientside_tag_service

BIN
static/testing/muh_apng.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 602 KiB

19
test.py
View File

@ -19,6 +19,7 @@ from include import TestClientConstants
from include import TestClientDaemons
from include import TestClientDownloading
from include import TestClientListBoxes
from include import TestClientNetworking
from include import TestConstants
from include import TestDialogs
from include import TestDB
@ -125,8 +126,8 @@ class Controller( object ):
self._managers = {}
self._services_manager = ClientCaches.ServicesManager( self )
self._client_files_manager = ClientCaches.ClientFilesManager( self )
self.services_manager = ClientCaches.ServicesManager( self )
self.client_files_manager = ClientCaches.ClientFilesManager( self )
self._client_session_manager = ClientCaches.HydrusSessionManager( self )
self._managers[ 'tag_censorship' ] = ClientCaches.TagCensorshipManager( self )
@ -195,11 +196,6 @@ class Controller( object ):
def DoHTTP( self, *args, **kwargs ): return self._http.Request( *args, **kwargs )
def GetClientFilesManager( self ):
return self._client_files_manager
def GetClientSessionManager( self ):
return self._client_session_manager
@ -222,11 +218,9 @@ class Controller( object ):
return HC.options
def GetManager( self, manager_type ): return self._managers[ manager_type ]
def GetServicesManager( self ):
def GetManager( self, manager_type ):
return self._services_manager
return self._managers[ manager_type ]
def GetServerSessionManager( self ):
@ -293,8 +287,9 @@ class Controller( object ):
suites.append( unittest.TestLoader().loadTestsFromModule( TestHydrusTags ) )
if run_all or only_run == 'db': suites.append( unittest.TestLoader().loadTestsFromModule( TestDB ) )
if run_all or only_run == 'downloading':
suites.append( unittest.TestLoader().loadTestsFromModule( TestClientDownloading ) )
if run_all or only_run == 'networking':
suites.append( unittest.TestLoader().loadTestsFromModule( TestClientNetworking ) )
suites.append( unittest.TestLoader().loadTestsFromModule( TestHydrusNetworking ) )
if run_all or only_run == 'gui':
suites.append( unittest.TestLoader().loadTestsFromModule( TestDialogs ) )