RepoMirrors
/
hydrus
mirror of https://github.com/hydrusnetwork/hydrus.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Version 498

This commit is contained in:
Hydrus Network Developer 2022-08-31 16:19:53 -05:00
parent dabdbe2861
commit aa78ecaafc
14 changed files with 2300 additions and 493 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

56
docs/changelog.md
View File

@ -3,6 +3,38 @@
!!! note
This is the new changelog, only the most recent builds. For all versions, see the [old changelog](old_changelog.html).
## [Version 498](https://github.com/hydrusnetwork/hydrus/releases/tag/v498)
_almost all the changes this week are only important to server admins and janitors. regular users can skip updating this week_
## overview
* the server has important database and network updates this week. if your server has a lot of content, it has to count it all up, so it will take a short while to update. the petition protocol has also changed, so older clients will not be able to fetch new servers' petitions without an error. I think newer clients will be able to fetch older servers' ones, but it may be iffy
* I considered whether I should update the network protocol version number, which would (politely) force all users to update, but as this causes inconvenience every time I do it, and I expect to do more incremental updates here in coming weeks, and since this only affects admins and janitors, I decided to not. we are going to be in awkward flux for a little bit, so please make sure you update privileged clients and servers at roughly the same time
## server petition workflow
* the server now maintains an ongoing fast count of its various repository metadata, such as 'number of mappings' and 'number of petitions of type x'. when you fetch petition counts, no longer will it count live and max out at 1,000, it'll give you good full numbers every time, and real fast
* you can see the current numbers from the new 'service info' button on review services, which only appears in advanced mode. any user with an account key can see these numbers, which include number of petitions in the queue. I can make this more private if you like, but for now I think it is good if advanced users can see them all
* in the petition processing page, sibling and parent petitions will now include both delete and add rows if the account and reason are the same. I'm aiming to get better 'full' coverage of a replace petition, so you can see and approve/deny both the add and the remove parts in one go. for fetching, these combined petitions count as 'delete' petitions, and won't appear in the 'add' petition queue
* when users encounter an automatic conflict resolution in the manage siblings dialog, those auto-petitioned pairs are now assigned the same reason as the original conflicting pended pairs. they _should_ show up together in the new petition processing UI
* as part of this, sibling and parent petitions are no longer filtered by namespace. you will see everything with that same account and reason in one go. let's try it out, and if it is too much, I will add filters clientside or something. since we are now starting to see add and remove together, we'll want to at least have the option to see everything
## boring server stuff
* the petition object is updated to handle multiple actions per petition, and the clientside petition UI is updated appropriately
* the server tracks 'actionable' petition counts as separate to the number of raw petition rows. some of this was happening before, but the logic is improved, including clever counting of the new petitions that include both add and delete rows
* for when my count-update logic inevitably fails, there is now a 'regen service info' entry in the 'administrate services' menu for all repositories. numbers generated will be printed to server log
* some unusual repo upload logic is cleaned up, e.g. if a user with 'create permission' uploads a sibling or parent, any pending rows for that content will now be properly cleared)
* fixed a stupid swap logical bug where janitors who could only moderate siblings (and not parents) were only being given parent numbers and vice versa
* all server services now respond to /busy check. it requires no authentication and just returns 1 or 0 depending on the current lock state
* fixed a bug where tag siblings or parents that were denied would still make a new definition record for the child/bad tag
* with all the fine number changes, fleshed out the server unit tests with more examples of submitting and altering content and then checking for numbers afterwards. now checked are: file add, file admin delete, mapping add, mapping admin delete, mapping petition, mapping petition approve+deny, parent add, parent admin delete, parent pend, parent pend approve+deny, parent petition, parent petition approve+deny
* significant refactoring of the tail end of server content update pipeline. more things now go through logic-harmonised update methods that ensure count is reliable
* did some misc server db and constant enum code cleanup
## misc
* to match the new change in the server, in the client, tag and rating services now store their 'num_files' service info count as the new 'num_file_hashes'. existing numbers will be converted over during update
* fixed a probably ten year old bug where 'num pending/petitioned files' had the same enum as 'num pending/petitioned mappings'. never noticed, since no service has done both those things
* if the upload pending process fails due to an unusual permission error or similar, the pending menu should now recover and update itself (previously it stayed greyed out)
## [Version 497](https://github.com/hydrusnetwork/hydrus/releases/tag/v497)
### misc
@ -319,27 +351,3 @@
* may have fixed an issue with a very slow to boot client trying to politely wait on the thumbnail cache before it instantiates
* misc UI text rewording and layout flag fixes
* fixed some jank formatting on database migration help
## [Version 487](https://github.com/hydrusnetwork/hydrus/releases/tag/v487)
### misc
* updated the duplicate filter 'show next pair' logic again, mostly simplification and merging of decision making. it _should_ be even more resistant to weird problems at the end of batches, particularly if you have deleted files manually
* a new button on the duplicate filter right hover window now appends the current pair to the parent duplicate media page (for if you want to do more processing to them later)
* if you manually delete a file in the duplicate filter, if that file appears again in the current batch of pairs, those will be auto-skipped
* if you manually delete a file in the duplicate filter, the actual delete is now deferred to when you commit the batch! it also undoes if you go back!
* fixed a bug when editing the external program launch paths in the options
* fixed an annoying delay-and-error-popup when clearing the separator field when editing a String Splitter. now the field just turns red and vetoes an OK with a nicer error text
* also improved how string splitters report actual split errors
* if you are in advanced mode, the _review services_ panels now have an 'id' button that lets you fetch the database service id
* wrote a new database maintenance routine under _database->check and repair->resync tag mappings cache files_, which is a lightweight way of fixing ghost files or situations where files with a tag are neither counted nor appear in file results. this fixes these problems in a couple minutes, so for this it is much better than a full regen of the cache
### cleanup and other boring stuff
* the archive/delete filter now says which file domain it will be deleting from
* if an archive/delete filter is launched on a 'multiple locations' file domain, it is now careful to only make delete records for the deleted files for the file services each one is actually in
* renamed the 'default local file search location' option to 'fallback' and updated its tooltip a bit. this was really a hacky thing I needed to fill some gaps while rewriting from 'my files' to multiple local file services. the whole thing needs more attention to become more useful. I also fixed an issue where it could become invalid 'nothing' if you deleted a file service it was referring to (issue #1155)
* I think I fixed a rare 'did not find info for that file' style problem when highlighting some watchers/downloaders
* I think I have silenced some unhelpful BeautifulSoup (html parser) warnings that were spamming to the log in some situations
* updated last week's big update to work with TRUNCATE journalling mode. I will be doing this for other big updates going forwards, since multi-GB WAL transactions cause problems for some users
* last week's update also gives a time estimate in its pre-popup, based on 60k files per minute
* removed some old database cache data that wasn't cleared in a previous update
* a variety of misc UI text fixes and cleanup

31
docs/old_changelog.html
View File

@ -33,6 +33,37 @@
<div class="content">
<h3 id="changelog"><a href="#changelog">changelog</a></h3>
<ul>
<li><h3 id="version_498"><a href="#version_498">version 498</a></h3></li>
<ul>
<li>_almost all the changes this week are only important to server admins and janitors. regular users can skip updating this week_</li>
<li>overview:</li>
<li>the server has important database and network updates this week. if your server has a lot of content, it has to count it all up, so it will take a short while to update. the petition protocol has also changed, so older clients will not be able to fetch new servers' petitions without an error. I think newer clients will be able to fetch older servers' ones, but it may be iffy</li>
<li>I considered whether I should update the network protocol version number, which would (politely) force all users to update, but as this causes inconvenience every time I do it, and I expect to do more incremental updates here in coming weeks, and since this only affects admins and janitors, I decided to not. we are going to be in awkward flux for a little bit, so please make sure you update privileged clients and servers at roughly the same time</li>
<li>.</li>
<li>server petition workflow:</li>
<li>the server now maintains an ongoing fast count of its various repository metadata, such as 'number of mappings' and 'number of petitions of type x'. when you fetch petition counts, no longer will it count live and max out at 1,000, it'll give you good full numbers every time, and real fast</li>
<li>you can see the current numbers from the new 'service info' button on review services, which only appears in advanced mode. any user with an account key can see these numbers, which include number of petitions in the queue. I can make this more private if you like, but for now I think it is good if advanced users can see them all</li>
<li>in the petition processing page, sibling and parent petitions will now include both delete and add rows if the account and reason are the same. I'm aiming to get better 'full' coverage of a replace petition, so you can see and approve/deny both the add and the remove parts in one go. for fetching, these combined petitions count as 'delete' petitions, and won't appear in the 'add' petition queue</li>
<li>when users encounter an automatic conflict resolution in the manage siblings dialog, those auto-petitioned pairs are now assigned the same reason as the original conflicting pended pairs. they _should_ show up together in the new petition processing UI</li>
<li>as part of this, sibling and parent petitions are no longer filtered by namespace. you will see everything with that same account and reason in one go. let's try it out, and if it is too much, I will add filters clientside or something. since we are now starting to see add and remove together, we'll want to at least have the option to see everything</li>
<li>.</li>
<li>boring server stuff:</li>
<li>the petition object is updated to handle multiple actions per petition, and the clientside petition UI is updated appropriately</li>
<li>the server tracks 'actionable' petition counts as separate to the number of raw petition rows. some of this was happening before, but the logic is improved, including clever counting of the new petitions that include both add and delete rows</li>
<li>for when my count-update logic inevitably fails, there is now a 'regen service info' entry in the 'administrate services' menu for all repositories. numbers generated will be printed to server log</li>
<li>some unusual repo upload logic is cleaned up, e.g. if a user with 'create permission' uploads a sibling or parent, any pending rows for that content will now be properly cleared)</li>
<li>fixed a stupid swap logical bug where janitors who could only moderate siblings (and not parents) were only being given parent numbers and vice versa</li>
<li>all server services now respond to /busy check. it requires no authentication and just returns 1 or 0 depending on the current lock state</li>
<li>fixed a bug where tag siblings or parents that were denied would still make a new definition record for the child/bad tag</li>
<li>with all the fine number changes, fleshed out the server unit tests with more examples of submitting and altering content and then checking for numbers afterwards. now checked are: file add, file admin delete, mapping add, mapping admin delete, mapping petition, mapping petition approve+deny, parent add, parent admin delete, parent pend, parent pend approve+deny, parent petition, parent petition approve+deny</li>
<li>significant refactoring of the tail end of server content update pipeline. more things now go through logic-harmonised update methods that ensure count is reliable</li>
<li>did some misc server db and constant enum code cleanup</li>
<li>.</li>
<li>misc:</li>
<li>to match the new change in the server, in the client, tag and rating services now store their 'num_files' service info count as the new 'num_file_hashes'. existing numbers will be converted over during update</li>
<li>fixed a probably ten year old bug where 'num pending/petitioned files' had the same enum as 'num pending/petitioned mappings'. never noticed, since no service has done both those things</li>
<li>if the upload pending process fails due to an unusual permission error or similar, the pending menu should now recover and update itself (previously it stayed greyed out)</li>
</ul>
<li><h3 id="version_497"><a href="#version_497">version 497</a></h3></li>
<ul>
<li>misc:</li>

73
hydrus/client/db/ClientDB.py
View File

@ -3182,11 +3182,13 @@ class DB( HydrusDB.HydrusDB ):
service_type = self.modules_services.GetServiceType( service_id )
service_info = self._GetServiceInfoSpecific( service_id, service_type, { HC.SERVICE_INFO_NUM_FILES }, calculate_missing = False )
info_type = HC.SERVICE_INFO_NUM_FILE_HASHES
if HC.SERVICE_INFO_NUM_FILES in service_info:
service_info = self._GetServiceInfoSpecific( service_id, service_type, { info_type }, calculate_missing = False )
if info_type in service_info:
num_everything = service_info[ HC.SERVICE_INFO_NUM_FILES ]
num_everything = service_info[ info_type ]
system_everythings.append( ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_EVERYTHING, count = ClientSearch.PredicateCount.STATICCreateCurrentCount( num_everything ) ) )
@ -5753,15 +5755,15 @@ class DB( HydrusDB.HydrusDB ):
elif service_type == HC.LOCAL_TAG:
info_types = { HC.SERVICE_INFO_NUM_FILES, HC.SERVICE_INFO_NUM_TAGS, HC.SERVICE_INFO_NUM_MAPPINGS }
info_types = { HC.SERVICE_INFO_NUM_FILE_HASHES, HC.SERVICE_INFO_NUM_TAGS, HC.SERVICE_INFO_NUM_MAPPINGS }
elif service_type == HC.TAG_REPOSITORY:
info_types = { HC.SERVICE_INFO_NUM_FILES, HC.SERVICE_INFO_NUM_TAGS, HC.SERVICE_INFO_NUM_MAPPINGS, HC.SERVICE_INFO_NUM_DELETED_MAPPINGS }
info_types = { HC.SERVICE_INFO_NUM_FILE_HASHES, HC.SERVICE_INFO_NUM_TAGS, HC.SERVICE_INFO_NUM_MAPPINGS, HC.SERVICE_INFO_NUM_DELETED_MAPPINGS }
elif service_type in ( HC.LOCAL_RATING_LIKE, HC.LOCAL_RATING_NUMERICAL ):
info_types = { HC.SERVICE_INFO_NUM_FILES }
info_types = { HC.SERVICE_INFO_NUM_FILE_HASHES }
elif service_type == HC.LOCAL_BOORU:
@ -5839,7 +5841,7 @@ class DB( HydrusDB.HydrusDB ):
save_it = False
if info_type == HC.SERVICE_INFO_NUM_FILES:
if info_type == HC.SERVICE_INFO_NUM_FILE_HASHES:
info = self.modules_mappings_storage.GetCurrentFilesCount( service_id )
@ -5888,7 +5890,7 @@ class DB( HydrusDB.HydrusDB ):
elif service_type in ( HC.LOCAL_RATING_LIKE, HC.LOCAL_RATING_NUMERICAL ):
if info_type == HC.SERVICE_INFO_NUM_FILES:
if info_type == HC.SERVICE_INFO_NUM_FILE_HASHES:
( info, ) = self._Execute( 'SELECT COUNT( * ) FROM local_ratings WHERE service_id = ?;', ( service_id, ) ).fetchone()
@ -6584,7 +6586,7 @@ class DB( HydrusDB.HydrusDB ):
our_results = self._GetServiceInfo( tag_service_key )
our_num_files = our_results[ HC.SERVICE_INFO_NUM_FILES ]
our_num_files = our_results[ HC.SERVICE_INFO_NUM_FILE_HASHES ]
other_services = [ service for service in self.modules_services.GetServices( HC.REAL_TAG_SERVICES ) if service.GetServiceKey() != tag_service_key ]
@ -6594,7 +6596,7 @@ class DB( HydrusDB.HydrusDB ):
other_results = self._GetServiceInfo( other_service.GetServiceKey() )
other_num_files.append( other_results[ HC.SERVICE_INFO_NUM_FILES ] )
other_num_files.append( other_results[ HC.SERVICE_INFO_NUM_FILE_HASHES ] )
if len( other_num_files ) == 0:
@ -7379,7 +7381,7 @@ class DB( HydrusDB.HydrusDB ):
ratings_added += self._GetRowCount()
self._Execute( 'UPDATE service_info SET info = info + ? WHERE service_id = ? AND info_type = ?;', ( ratings_added, service_id, HC.SERVICE_INFO_NUM_FILES ) )
self._Execute( 'UPDATE service_info SET info = info + ? WHERE service_id = ? AND info_type = ?;', ( ratings_added, service_id, HC.SERVICE_INFO_NUM_FILE_HASHES ) )
elif action == HC.CONTENT_UPDATE_ADVANCED:
@ -7394,7 +7396,7 @@ class DB( HydrusDB.HydrusDB ):
ratings_deleted = self._GetRowCount()
self._Execute( 'UPDATE service_info SET info = info - ? WHERE service_id = ? AND info_type = ?;', ( ratings_deleted, service_id, HC.SERVICE_INFO_NUM_FILES ) )
self._Execute( 'UPDATE service_info SET info = info - ? WHERE service_id = ? AND info_type = ?;', ( ratings_deleted, service_id, HC.SERVICE_INFO_NUM_FILE_HASHES ) )
elif action == 'delete_for_non_local_files':
@ -7404,13 +7406,13 @@ class DB( HydrusDB.HydrusDB ):
ratings_deleted = self._GetRowCount()
self._Execute( 'UPDATE service_info SET info = info - ? WHERE service_id = ? AND info_type = ?;', ( ratings_deleted, service_id, HC.SERVICE_INFO_NUM_FILES ) )
self._Execute( 'UPDATE service_info SET info = info - ? WHERE service_id = ? AND info_type = ?;', ( ratings_deleted, service_id, HC.SERVICE_INFO_NUM_FILE_HASHES ) )
elif action == 'delete_for_all_files':
self._Execute( 'DELETE FROM local_ratings WHERE service_id = ?;', ( service_id, ) )
self._Execute( 'UPDATE service_info SET info = ? WHERE service_id = ? AND info_type = ?;', ( 0, service_id, HC.SERVICE_INFO_NUM_FILES ) )
self._Execute( 'UPDATE service_info SET info = ? WHERE service_id = ? AND info_type = ?;', ( 0, service_id, HC.SERVICE_INFO_NUM_FILE_HASHES ) )
@ -9251,7 +9253,7 @@ class DB( HydrusDB.HydrusDB ):
if HC.CONTENT_TYPE_MAPPINGS in content_types:
service_info_types_to_delete.extend( { HC.SERVICE_INFO_NUM_FILES, HC.SERVICE_INFO_NUM_TAGS, HC.SERVICE_INFO_NUM_MAPPINGS, HC.SERVICE_INFO_NUM_DELETED_MAPPINGS } )
service_info_types_to_delete.extend( { HC.SERVICE_INFO_NUM_FILE_HASHES, HC.SERVICE_INFO_NUM_TAGS, HC.SERVICE_INFO_NUM_MAPPINGS, HC.SERVICE_INFO_NUM_DELETED_MAPPINGS } )
if service_type in HC.REAL_TAG_SERVICES:
@ -11477,6 +11479,45 @@ class DB( HydrusDB.HydrusDB ):
if version == 497:
try:
file_service_ids = self.modules_services.GetServiceIds( HC.FILE_SERVICES )
# updating some borked enums that were overwriting tag enums
for service_id in file_service_ids:
self._Execute( 'UPDATE service_info SET info_type = ? WHERE service_id = ? AND info_type = ?', ( HC.SERVICE_INFO_NUM_PENDING_FILES, service_id, 15 ) )
self._Execute( 'UPDATE service_info SET info_type = ? WHERE service_id = ? AND info_type = ?', ( HC.SERVICE_INFO_NUM_PETITIONED_FILES, service_id, 16 ) )
tag_service_ids = self.modules_services.GetServiceIds( HC.ALL_TAG_SERVICES )
# moving 'file count' to 'file hash count'
for service_id in tag_service_ids:
self._Execute( 'UPDATE service_info SET info_type = ? WHERE service_id = ? AND info_type = ?', ( HC.SERVICE_INFO_NUM_FILE_HASHES, service_id, HC.SERVICE_INFO_NUM_FILES ) )
rating_service_ids = self.modules_services.GetServiceIds( HC.RATINGS_SERVICES )
# moving 'file count' to 'file hash count'
for service_id in rating_service_ids:
self._Execute( 'UPDATE service_info SET info_type = ? WHERE service_id = ? AND info_type = ?', ( HC.SERVICE_INFO_NUM_FILE_HASHES, service_id, HC.SERVICE_INFO_NUM_FILES ) )
except Exception as e:
HydrusData.PrintException( e )
message = 'Trying to update some cached numbers failed! Please let hydrus dev know!'
self.pub_initial_message( message )
self._controller.frame_splash_status.SetTitleText( 'updated db to v{}'.format( HydrusData.ToHumanInt( version + 1 ) ) )
self._Execute( 'UPDATE version SET version = ?;', ( version + 1, ) )
@ -11695,7 +11736,7 @@ class DB( HydrusDB.HydrusDB ):
if change_in_num_deleted_mappings != 0: service_info_updates.append( ( change_in_num_deleted_mappings, tag_service_id, HC.SERVICE_INFO_NUM_DELETED_MAPPINGS ) )
if change_in_num_pending_mappings != 0: service_info_updates.append( ( change_in_num_pending_mappings, tag_service_id, HC.SERVICE_INFO_NUM_PENDING_MAPPINGS ) )
if change_in_num_petitioned_mappings != 0: service_info_updates.append( ( change_in_num_petitioned_mappings, tag_service_id, HC.SERVICE_INFO_NUM_PETITIONED_MAPPINGS ) )
if change_in_num_files != 0: service_info_updates.append( ( change_in_num_files, tag_service_id, HC.SERVICE_INFO_NUM_FILES ) )
if change_in_num_files != 0: service_info_updates.append( ( change_in_num_files, tag_service_id, HC.SERVICE_INFO_NUM_FILE_HASHES ) )
if len( service_info_updates ) > 0: self._ExecuteMany( 'UPDATE service_info SET info = info + ? WHERE service_id = ? AND info_type = ?;', service_info_updates )

195
hydrus/client/gui/ClientGUI.py
View File

@ -418,6 +418,8 @@ def THREADUploadPending( service_key ):
finally:
HG.client_controller.pub( 'notify_pending_upload_finished', service_key )
HydrusData.Print( job_key.ToString() )
job_key.Finish()
@ -455,8 +457,6 @@ def THREADUploadPending( service_key ):
HG.client_controller.Write( 'delete_service_info', service_key, types_to_delete )
HG.client_controller.pub( 'notify_pending_upload_finished', service_key )
class FrameGUI( ClientGUITopLevelWindows.MainFrameThatResizes, CAC.ApplicationCommandProcessorMixin ):
@ -2917,6 +2917,10 @@ class FrameGUI( ClientGUITopLevelWindows.MainFrameThatResizes, CAC.ApplicationCo
ClientGUIMenus.AppendMenuItem( submenu, 'change anonymisation period', 'Change the account history nullification period for this service.', self._ManageServiceOptionsNullificationPeriod, service_key )
ClientGUIMenus.AppendSeparator( submenu )
ClientGUIMenus.AppendMenuItem( submenu, 'maintenance: regen service info', 'Add, edit, and delete this server\'s services.', self._ServerMaintenanceRegenServiceInfo, service_key )
if can_overrule_services and service_type == HC.SERVER_ADMIN:
@ -3378,7 +3382,8 @@ class FrameGUI( ClientGUITopLevelWindows.MainFrameThatResizes, CAC.ApplicationCo
ClientGUIMenus.AppendMenuItem( tests, 'run the ui test', 'Run hydrus_dev\'s weekly UI Test. Guaranteed to work and not mess up your session, ha ha.', self._RunUITest )
ClientGUIMenus.AppendMenuItem( tests, 'run the client api test', 'Run hydrus_dev\'s weekly Client API Test. Guaranteed to work and not mess up your session, ha ha.', self._RunClientAPITest )
ClientGUIMenus.AppendMenuItem( tests, 'run the server test', 'This will try to boot the server in your install folder and initialise it. This is mostly here for testing purposes.', self._RunServerTest )
ClientGUIMenus.AppendMenuItem( tests, 'run the server test', 'This will try to boot the server in your install folder and initialise it.', self._RunServerTest )
ClientGUIMenus.AppendMenuItem( tests, 'run the server test on fresh server', 'This will try to initialise an already running server.', self._RunServerTest, do_boot = False )
ClientGUIMenus.AppendMenu( debug, tests, 'tests, do not touch' )
@ -6047,85 +6052,88 @@ class FrameGUI( ClientGUITopLevelWindows.MainFrameThatResizes, CAC.ApplicationCo
HG.client_controller.CallToThread( do_it )
def _RunServerTest( self ):
def _RunServerTest( self, do_boot = True ):
def do_it():
host = '127.0.0.1'
port = HC.DEFAULT_SERVER_ADMIN_PORT
if HydrusNetworking.LocalPortInUse( port ):
if do_boot:
HydrusData.ShowText( 'The server appears to be already running. Either that, or something else is using port ' + str( HC.DEFAULT_SERVER_ADMIN_PORT ) + '.' )
return
else:
try:
if HydrusNetworking.LocalPortInUse( port ):
HydrusData.ShowText( 'Starting server\u2026' )
db_param = '-d=' + self._controller.GetDBDir()
if HC.PLATFORM_WINDOWS:
server_frozen_path = os.path.join( HC.BASE_DIR, 'server.exe' )
else:
server_frozen_path = os.path.join( HC.BASE_DIR, 'server' )
if os.path.exists( server_frozen_path ):
cmd = [ server_frozen_path, db_param ]
else:
python_executable = sys.executable
if python_executable.endswith( 'client.exe' ) or python_executable.endswith( 'client' ):
raise Exception( 'Could not automatically set up the server--could not find server executable or python executable.' )
if 'pythonw' in python_executable:
python_executable = python_executable.replace( 'pythonw', 'python' )
server_script_path = os.path.join( HC.BASE_DIR, 'server.py' )
cmd = [ python_executable, server_script_path, db_param ]
sbp_kwargs = HydrusData.GetSubprocessKWArgs( hide_terminal = False )
HydrusData.CheckProgramIsNotShuttingDown()
subprocess.Popen( cmd, **sbp_kwargs )
time_waited = 0
while not HydrusNetworking.LocalPortInUse( port ):
time.sleep( 3 )
time_waited += 3
if time_waited > 30:
raise Exception( 'The server\'s port did not appear!' )
except:
HydrusData.ShowText( 'I tried to start the server, but something failed!' + os.linesep + traceback.format_exc() )
HydrusData.ShowText( 'The server appears to be already running. Either that, or something else is using port ' + str( HC.DEFAULT_SERVER_ADMIN_PORT ) + '.' )
return
else:
try:
HydrusData.ShowText( 'Starting server\u2026' )
db_param = '-d=' + self._controller.GetDBDir()
if HC.PLATFORM_WINDOWS:
server_frozen_path = os.path.join( HC.BASE_DIR, 'server.exe' )
else:
server_frozen_path = os.path.join( HC.BASE_DIR, 'server' )
if os.path.exists( server_frozen_path ):
cmd = [ server_frozen_path, db_param ]
else:
python_executable = sys.executable
if python_executable.endswith( 'client.exe' ) or python_executable.endswith( 'client' ):
raise Exception( 'Could not automatically set up the server--could not find server executable or python executable.' )
if 'pythonw' in python_executable:
python_executable = python_executable.replace( 'pythonw', 'python' )
server_script_path = os.path.join( HC.BASE_DIR, 'server.py' )
cmd = [ python_executable, server_script_path, db_param ]
sbp_kwargs = HydrusData.GetSubprocessKWArgs( hide_terminal = False )
HydrusData.CheckProgramIsNotShuttingDown()
subprocess.Popen( cmd, **sbp_kwargs )
time_waited = 0
while not HydrusNetworking.LocalPortInUse( port ):
time.sleep( 3 )
time_waited += 3
if time_waited > 30:
raise Exception( 'The server\'s port did not appear!' )
except:
HydrusData.ShowText( 'I tried to start the server, but something failed!' + os.linesep + traceback.format_exc() )
return
time.sleep( 5 )
@ -6206,7 +6214,7 @@ class FrameGUI( ClientGUITopLevelWindows.MainFrameThatResizes, CAC.ApplicationCo
HydrusData.ShowText( 'Done! Check services->review services to see your new server and its services.' )
text = 'This will attempt to start the server in the same install directory as this client, initialise it, and store the resultant admin accounts in the client.'
text = 'Woe unto you unless you click "no" NOW.'
result = ClientGUIDialogsQuick.GetYesNo( self, text )
@ -6226,6 +6234,49 @@ class FrameGUI( ClientGUITopLevelWindows.MainFrameThatResizes, CAC.ApplicationCo
def _ServerMaintenanceRegenServiceInfo( self, service_key: bytes ):
def do_it( service ):
started = HydrusData.GetNow()
service.Request( HC.POST, 'maintenance_regen_service_info' )
HydrusData.ShowText( 'Maintenance started!' )
time.sleep( 10 )
result_bytes = service.Request( HC.GET, 'busy' )
while result_bytes == b'1':
if HG.started_shutdown:
return
time.sleep( 10 )
result_bytes = service.Request( HC.GET, 'busy' )
it_took = HydrusData.GetNow() - started
HydrusData.ShowText( 'Server maintenance done in ' + HydrusData.TimeDeltaToPrettyTimeDelta( it_took ) + '!' )
message = 'This will tell the server to recalculate the cached numbers for number of files, mappings, actionable petitions and so on. It may take a little while to complete, during which time it will not be able to serve any requests.'
result = ClientGUIDialogsQuick.GetYesNo( self, message, yes_label = 'do it', no_label = 'forget it' )
if result == QW.QDialog.Accepted:
service = self._controller.services_manager.GetService( service_key )
self._controller.CallToThread( do_it, service )
def _SetPassword( self ):
message = '''You can set a password to be asked for whenever the client starts.

62
hydrus/client/gui/ClientGUITags.py
View File

@ -3085,7 +3085,7 @@ class ManageTagParents( ClientGUIScrolledPanels.ManagePanel ):
affected_pairs = []
if len( new_pairs ) > 0:
do_it = True
if not self._i_am_local_tag_service:
@ -3127,7 +3127,10 @@ class ManageTagParents( ClientGUIScrolledPanels.ManagePanel ):
if do_it:
for pair in new_pairs: self._pairs_to_reasons[ pair ] = reason
for pair in new_pairs:
self._pairs_to_reasons[ pair ] = reason
@ -3199,7 +3202,10 @@ class ManageTagParents( ClientGUIScrolledPanels.ManagePanel ):
if do_it:
for pair in current_pairs: self._pairs_to_reasons[ pair ] = reason
for pair in current_pairs:
self._pairs_to_reasons[ pair ] = reason
@ -3911,6 +3917,8 @@ class ManageTagSiblings( ClientGUIScrolledPanels.ManagePanel ):
class _Panel( QW.QWidget ):
AUTO_PETITION_REASON = 'TO BE AUTO-PETITIONED'
def __init__( self, parent, service_key, tags = None ):
QW.QWidget.__init__( self, parent )
@ -4134,7 +4142,28 @@ class ManageTagSiblings( ClientGUIScrolledPanels.ManagePanel ):
if do_it:
for pair in new_pairs: self._pairs_to_reasons[ pair ] = reason
we_are_autopetitioning = self.AUTO_PETITION_REASON in self._pairs_to_reasons.values()
if we_are_autopetitioning:
reason = 'REPLACEMENT: {}'.format( reason )
for pair in new_pairs:
self._pairs_to_reasons[ pair ] = reason
if we_are_autopetitioning:
for ( p, r ) in list( self._pairs_to_reasons.items() ):
if r == self.AUTO_PETITION_REASON:
self._pairs_to_reasons[ p ] = reason
@ -4196,11 +4225,26 @@ class ManageTagSiblings( ClientGUIScrolledPanels.ManagePanel ):
if do_it:
if reason is not None:
we_are_autopetitioning = self.AUTO_PETITION_REASON in self._pairs_to_reasons.values()
if we_are_autopetitioning:
for pair in current_pairs:
reason = 'REPLACEMENT: {}'.format( reason )
for pair in current_pairs:
self._pairs_to_reasons[ pair ] = reason
if we_are_autopetitioning:
for ( p, r ) in list( self._pairs_to_reasons.items() ):
self._pairs_to_reasons[ pair ] = reason
if r == self.AUTO_PETITION_REASON:
self._pairs_to_reasons[ p ] = reason
@ -4295,7 +4339,7 @@ class ManageTagSiblings( ClientGUIScrolledPanels.ManagePanel ):
pairs_to_auto_petition = list( pairs_to_auto_petition )
self._AddPairs( pairs_to_auto_petition, remove_only = True, default_reason = 'AUTO-PETITION TO REASSIGN TO: ' + new )
self._AddPairs( pairs_to_auto_petition, remove_only = True, default_reason = self.AUTO_PETITION_REASON )
@ -4321,7 +4365,7 @@ class ManageTagSiblings( ClientGUIScrolledPanels.ManagePanel ):
pairs_to_auto_petition = [ ( loop_new, next_new ) ]
self._AddPairs( pairs_to_auto_petition, remove_only = True, default_reason = 'AUTO-PETITION TO BREAK LOOP FOR: {}->{}'.format( potential_old, potential_new ) )
self._AddPairs( pairs_to_auto_petition, remove_only = True, default_reason = self.AUTO_PETITION_REASON )
current_pairs = self._current_statuses_to_pairs[ HC.CONTENT_STATUS_CURRENT ].union( self._current_statuses_to_pairs[ HC.CONTENT_STATUS_PENDING ] ).difference( self._current_statuses_to_pairs[ HC.CONTENT_STATUS_PETITIONED ] )

403
hydrus/client/gui/pages/ClientGUIManagement.py
View File

@ -4344,12 +4344,19 @@ class ManagementPanelPetitions( ManagementPanel ):
self._sort_by_left.setEnabled( False )
self._sort_by_right.setEnabled( False )
self._contents = ClientGUICommon.BetterCheckBoxList( self._petition_panel )
self._contents.itemDoubleClicked.connect( self.EventContentDoubleClick )
self._contents_add = ClientGUICommon.BetterCheckBoxList( self._petition_panel )
self._contents_add.itemDoubleClicked.connect( self.ContentsAddDoubleClick )
( min_width, min_height ) = ClientGUIFunctions.ConvertTextToPixels( self._contents, ( 16, 20 ) )
( min_width, min_height ) = ClientGUIFunctions.ConvertTextToPixels( self._contents_add, ( 16, 20 ) )
self._contents.setMinimumHeight( min_height )
self._contents_add.setMinimumHeight( min_height )
self._contents_delete = ClientGUICommon.BetterCheckBoxList( self._petition_panel )
self._contents_delete.itemDoubleClicked.connect( self.ContentsDeleteDoubleClick )
( min_width, min_height ) = ClientGUIFunctions.ConvertTextToPixels( self._contents_delete, ( 16, 20 ) )
self._contents_delete.setMinimumHeight( min_height )
self._process = QW.QPushButton( 'process', self._petition_panel )
self._process.clicked.connect( self.EventProcess )
@ -4388,7 +4395,8 @@ class ManagementPanelPetitions( ManagementPanel ):
self._petition_panel.Add( self._reason_text, CC.FLAGS_EXPAND_PERPENDICULAR )
self._petition_panel.Add( check_hbox, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
self._petition_panel.Add( sort_hbox, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
self._petition_panel.Add( self._contents, CC.FLAGS_EXPAND_BOTH_WAYS )
self._petition_panel.Add( self._contents_add, CC.FLAGS_EXPAND_BOTH_WAYS )
self._petition_panel.Add( self._contents_delete, CC.FLAGS_EXPAND_BOTH_WAYS )
self._petition_panel.Add( self._process, CC.FLAGS_EXPAND_PERPENDICULAR )
self._petition_panel.Add( self._copy_account_key_button, CC.FLAGS_EXPAND_PERPENDICULAR )
self._petition_panel.Add( self._modify_petitioner, CC.FLAGS_EXPAND_PERPENDICULAR )
@ -4414,24 +4422,35 @@ class ManagementPanelPetitions( ManagementPanel ):
self.widget().setLayout( vbox )
self._contents.rightClicked.connect( self.EventRowRightClick )
self._contents_add.rightClicked.connect( self.EventAddRowRightClick )
self._contents_delete.rightClicked.connect( self.EventDeleteRowRightClick )
self._DrawCurrentPetition()
def _CheckAll( self ):
for i in range( self._contents.count() ):
for i in range( self._contents_add.count() ):
self._contents.Check( i, True )
self._contents_add.Check( i, True )
for i in range( self._contents_delete.count() ):
self._contents_delete.Check( i, True )
def _CheckNone( self ):
for i in range( self._contents.count() ):
for i in range( self._contents_add.count() ):
self._contents.Check( i, False )
self._contents_add.Check( i, False )
for i in range( self._contents_delete.count() ):
self._contents_delete.Check( i, False )
@ -4457,7 +4476,12 @@ class ManagementPanelPetitions( ManagementPanel ):
self._reason_text.clear()
self._reason_text.setProperty( 'hydrus_text', 'default' )
self._contents.clear()
self._contents_add.clear()
self._contents_delete.clear()
self._contents_add.hide()
self._contents_delete.hide()
self._process.setEnabled( False )
self._copy_account_key_button.setEnabled( False )
@ -4471,24 +4495,45 @@ class ManagementPanelPetitions( ManagementPanel ):
else:
( action_text, action ) = self._current_petition.GetActionTextAndAction()
add_contents = self._current_petition.GetContents( HC.CONTENT_UPDATE_PEND )
delete_contents = self._current_petition.GetContents( HC.CONTENT_UPDATE_PETITION )
if action == HC.CONTENT_UPDATE_PEND:
have_add = len( add_contents ) > 0
have_delete = len( delete_contents ) > 0
action_text = 'UNKNOWN'
hydrus_text = 'default'
object_name = 'normal'
if have_add or have_delete:
hydrus_text = 'valid'
elif action == HC.CONTENT_UPDATE_PETITION:
hydrus_text = 'invalid'
if have_add and have_delete:
action_text = 'REPLACE'
elif have_add:
action_text = 'ADD'
hydrus_text = 'valid'
object_name = 'HydrusValid'
else:
action_text = 'DELETE'
hydrus_text = 'invalid'
object_name = 'HydrusInvalid'
self._action_text.setText( action_text )
self._action_text.setProperty( 'hydrus_text', hydrus_text )
self._action_text.setObjectName( object_name )
#self._action_text.setProperty( 'hydrus_text', hydrus_text )
reason = self._current_petition.GetReason()
self._reason_text.setPlainText( reason )
self._reason_text.setProperty( 'hydrus_text', hydrus_text )
self._reason_text.setObjectName( object_name )
#self._reason_text.setProperty( 'hydrus_text', hydrus_text )
if self._last_petition_type_fetched[0] in ( HC.CONTENT_TYPE_TAG_SIBLINGS, HC.CONTENT_TYPE_TAG_PARENTS ):
@ -4501,11 +4546,16 @@ class ManagementPanelPetitions( ManagementPanel ):
self._sort_by_right.setEnabled( False )
contents = self._current_petition.GetContents()
self._contents_add.setVisible( have_add )
self._contents_delete.setVisible( have_delete )
contents_and_checks = [ ( c, True ) for c in contents ]
contents_and_checks = [ ( c, True ) for c in add_contents ]
self._SetContentsAndChecks( contents_and_checks, 'right' )
self._SetContentsAndChecks( HC.CONTENT_UPDATE_PEND, contents_and_checks, 'right' )
contents_and_checks = [ ( c, True ) for c in delete_contents ]
self._SetContentsAndChecks( HC.CONTENT_UPDATE_PETITION, contents_and_checks, 'right' )
self._process.setEnabled( True )
self._copy_account_key_button.setEnabled( True )
@ -4530,7 +4580,7 @@ class ManagementPanelPetitions( ManagementPanel ):
( st, button ) = self._petition_types_to_controls[ petition_type ]
st.setText( HydrusData.ToHumanInt(count)+' petitions' )
st.setText( HydrusData.ToHumanInt( count )+' petitions' )
if count > 0:
@ -4645,7 +4695,7 @@ class ManagementPanelPetitions( ManagementPanel ):
self._DrawCurrentPetition()
self._ShowHashes( [ ])
self._ShowHashes( [] )
def qt_done():
@ -4690,22 +4740,38 @@ class ManagementPanelPetitions( ManagementPanel ):
def _FlipSelected( self ):
for i in self._contents.GetSelectedIndices():
for i in self._contents_add.GetSelectedIndices():
flipped_state = not self._contents.IsChecked( i )
flipped_state = not self._contents_add.IsChecked( i )
self._contents.Check( i, flipped_state )
self._contents_add.Check( i, flipped_state )
for i in self._contents_delete.GetSelectedIndices():
flipped_state = not self._contents_delete.IsChecked( i )
self._contents_delete.Check( i, flipped_state )
def _GetContentsAndChecks( self ):
def _GetContentsAndChecks( self, action ):
if action == HC.CONTENT_UPDATE_PEND:
contents = self._contents_add
else:
contents = self._contents_delete
contents_and_checks = []
for i in range( self._contents.count() ):
for i in range( contents.count() ):
content = self._contents.GetData( i )
check = self._contents.IsChecked( i )
content = contents.GetData( i )
check = contents.IsChecked( i )
contents_and_checks.append( ( content, check ) )
@ -4713,7 +4779,7 @@ class ManagementPanelPetitions( ManagementPanel ):
return contents_and_checks
def _SetContentsAndChecks( self, contents_and_checks, sort_type ):
def _SetContentsAndChecks( self, action, contents_and_checks, sort_type ):
def key( c_and_s ):
@ -4750,15 +4816,22 @@ class ManagementPanelPetitions( ManagementPanel ):
contents_and_checks.sort( key = key )
self._contents.clear()
if action == HC.CONTENT_UPDATE_PEND:
contents = self._contents_add
else:
contents = self._contents_delete
to_check = []
contents.clear()
for ( i, ( content, check ) ) in enumerate( contents_and_checks ):
content_string = content.ToString()
self._contents.Append( content_string, content, starts_checked = check )
contents.Append( content_string, content, starts_checked = check )
@ -4782,140 +4855,174 @@ class ManagementPanelPetitions( ManagementPanel ):
def _SortBy( self, sort_type ):
contents_and_checks = self._GetContentsAndChecks()
self._SetContentsAndChecks( contents_and_checks, sort_type )
for action in [ HC.CONTENT_UPDATE_PEND, HC.CONTENT_UPDATE_PETITION ]:
contents_and_checks = self._GetContentsAndChecks( action )
self._SetContentsAndChecks( action, contents_and_checks, sort_type )
def EventContentDoubleClick( self, item ):
def ContentsAddDoubleClick( self, item ):
selected_indices = self._contents.GetSelectedIndices()
selected_indices = self._contents_add.GetSelectedIndices()
if len( selected_indices ) > 0:
selection = selected_indices[0]
content = self._contents.GetData( selection )
content = self._contents_add.GetData( selection )
if content.HasHashes():
self.EventContentsDoubleClick( content )
def ContentsDeleteDoubleClick( self, item ):
selected_indices = self._contents_delete.GetSelectedIndices()
if len( selected_indices ) > 0:
selection = selected_indices[0]
content = self._contents_delete.GetData( selection )
self.EventContentsDoubleClick( content )
def EventContentsDoubleClick( self, content ):
if content.HasHashes():
hashes = content.GetHashes()
num_files_to_show = self._num_files_to_show.GetValue()
if num_files_to_show is not None and len( hashes ) > num_files_to_show:
hashes = content.GetHashes()
num_files_to_show = self._num_files_to_show.GetValue()
if num_files_to_show is not None and len( hashes ) > num_files_to_show:
hashes = random.sample( hashes, num_files_to_show )
self._ShowHashes( hashes )
hashes = random.sample( hashes, num_files_to_show )
self._ShowHashes( hashes )
def EventProcess( self ):
def break_approved_contents_into_chunks( approved_contents ):
def break_contents_into_chunks( some_contents ):
chunks_of_approved_contents = []
chunk_of_approved_contents = []
chunks_of_some_contents = []
chunk_of_some_contents = []
weight = 0
for content in approved_contents:
for content in some_contents:
for content_chunk in content.IterateUploadableChunks(): # break 20K-strong mappings petitions into smaller bits to POST back
chunk_of_approved_contents.append( content_chunk )
chunk_of_some_contents.append( content_chunk )
weight += content.GetVirtualWeight()
if weight > 50:
chunks_of_approved_contents.append( chunk_of_approved_contents )
chunks_of_some_contents.append( chunk_of_some_contents )
chunk_of_approved_contents = []
chunk_of_some_contents = []
weight = 0
if len( chunk_of_approved_contents ) > 0:
if len( chunk_of_some_contents ) > 0:
chunks_of_approved_contents.append( chunk_of_approved_contents )
chunks_of_some_contents.append( chunk_of_some_contents )
return chunks_of_approved_contents
return chunks_of_some_contents
def do_it( controller, service, petition_service_key, approved_contents, denied_contents, petition ):
def do_it( controller, service, petition_service_key, add_approved_contents, add_denied_contents, delete_approved_contents, delete_denied_contents, petition ):
jobs = [
( HC.CONTENT_UPDATE_PEND, True, add_approved_contents ),
( HC.CONTENT_UPDATE_PEND, False, add_denied_contents ),
( HC.CONTENT_UPDATE_PETITION, True, delete_approved_contents ),
( HC.CONTENT_UPDATE_PETITION, False, delete_denied_contents ),
]
num_done = 0
num_to_do = 0
for ( action, approved, contents ) in jobs:
num_to_do += len( contents )
if num_to_do > 1:
job_key = ClientThreading.JobKey( cancellable = True )
job_key.SetStatusTitle( 'committing petitions' )
HG.client_controller.pub( 'message', job_key )
else:
job_key = None
reason = petition.GetReason()
try:
num_done = 0
num_to_do = len( approved_contents )
if len( denied_contents ) > 0:
for ( action, approved, contents ) in jobs:
num_to_do += 1
if num_to_do > 1:
job_key = ClientThreading.JobKey( cancellable = True )
job_key.SetStatusTitle( 'committing petitions' )
HG.client_controller.pub( 'message', job_key )
else:
job_key = None
chunks_of_approved_contents = break_approved_contents_into_chunks( approved_contents )
num_approved_to_do = len( chunks_of_approved_contents )
for chunk_of_approved_contents in chunks_of_approved_contents:
if job_key is not None:
if len( contents ) == 0:
( i_paused, should_quit ) = job_key.WaitIfNeeded()
continue
if should_quit:
chunks_of_contents = break_contents_into_chunks( contents )
num_to_do += len( chunks_of_contents ) - 1
for chunk_of_contents in chunks_of_contents:
if job_key is not None:
return
( i_paused, should_quit ) = job_key.WaitIfNeeded()
if should_quit:
return
job_key.SetVariable( 'popup_gauge_1', ( num_done, num_to_do ) )
job_key.SetVariable( 'popup_gauge_1', ( num_done, num_approved_to_do ) )
content_updates = []
( update, content_updates ) = petition.GetApproval( chunk_of_approved_contents )
service.Request( HC.POST, 'update', { 'client_to_server_update' : update } )
controller.WriteSynchronous( 'content_updates', { petition_service_key : content_updates } )
num_done += 1
if len( denied_contents ) > 0:
if job_key is not None:
( i_paused, should_quit ) = job_key.WaitIfNeeded()
if should_quit:
if approved:
return
( update, content_updates ) = petition.GetApproval( action, chunk_of_contents, reason )
else:
update = petition.GetDenial( action, chunk_of_contents, reason )
update = petition.GetDenial( denied_contents )
service.Request( HC.POST, 'update', { 'client_to_server_update' : update } )
service.Request( HC.POST, 'update', { 'client_to_server_update' : update } )
if len( content_updates ) > 0:
controller.WriteSynchronous( 'content_updates', { petition_service_key : content_updates } )
num_done += 1
finally:
@ -4939,24 +5046,35 @@ class ManagementPanelPetitions( ManagementPanel ):
approved_contents = []
denied_contents = []
add_approved_contents = []
add_denied_contents = []
for index in range( self._contents.count() ):
delete_approved_contents = []
delete_denied_contents = []
jobs = [
( self._contents_add, add_approved_contents, add_denied_contents ),
( self._contents_delete, delete_approved_contents, delete_denied_contents )
]
for ( contents, approved_contents, denied_contents ) in jobs:
content = self._contents.GetData( index )
if self._contents.IsChecked( index ):
for index in range( contents.count() ):
approved_contents.append( content )
content = contents.GetData( index )
else:
denied_contents.append( content )
if contents.IsChecked( index ):
approved_contents.append( content )
else:
denied_contents.append( content )
HG.client_controller.CallToThread( do_it, self._controller, self._service, self._petition_service_key, approved_contents, denied_contents, self._current_petition )
HG.client_controller.CallToThread( do_it, self._controller, self._service, self._petition_service_key, add_approved_contents, add_denied_contents, delete_approved_contents, delete_denied_contents, self._current_petition )
self._current_petition = None
@ -4978,23 +5096,44 @@ class ManagementPanelPetitions( ManagementPanel ):
frame.SetPanel( panel )
def EventRowRightClick( self ):
def EventAddRowRightClick( self ):
selected_indices = self._contents.GetSelectedIndices()
selected_indices = self._contents_add.GetSelectedIndices()
selected_contents = []
for i in selected_indices:
content = self._contents.GetData( i )
content = self._contents_add.GetData( i )
selected_contents.append( content )
self.EventContentsRightClick( selected_contents )
def EventDeleteRowRightClick( self ):
selected_indices = self._contents_delete.GetSelectedIndices()
selected_contents = []
for i in selected_indices:
content = self._contents_delete.GetData( i )
selected_contents.append( content )
self.EventContentsRightClick( selected_contents )
def EventContentsRightClick( self, contents ):
copyable_items_a = []
copyable_items_b = []
for content in selected_contents:
for content in contents:
content_type = content.GetContentType()

66
hydrus/client/gui/services/ClientGUIClientsideServices.py
View File

@ -2632,6 +2632,8 @@ class ReviewServiceRepositorySubPanel( QW.QWidget ):
self._update_downloading_paused_button = ClientGUICommon.BetterBitmapButton( self._network_panel, CC.global_pixmaps().pause, self._PausePlayUpdateDownloading )
self._update_downloading_paused_button.setToolTip( 'pause/play update downloading' )
self._service_info_button = ClientGUICommon.BetterButton( self._network_panel, 'fetch service info', self._FetchServiceInfo )
self._sync_remote_now_button = ClientGUICommon.BetterButton( self._network_panel, 'download now', self._SyncRemoteNow )
reset_menu_items = []
@ -2705,6 +2707,7 @@ class ReviewServiceRepositorySubPanel( QW.QWidget ):
hbox = QP.HBoxLayout()
QP.AddToLayout( hbox, self._service_info_button, CC.FLAGS_CENTER_PERPENDICULAR )
QP.AddToLayout( hbox, self._sync_remote_now_button, CC.FLAGS_CENTER_PERPENDICULAR )
QP.AddToLayout( hbox, self._reset_downloading_button, CC.FLAGS_CENTER_PERPENDICULAR )
QP.AddToLayout( hbox, self._export_updates_button, CC.FLAGS_CENTER_PERPENDICULAR )
@ -2874,6 +2877,58 @@ class ReviewServiceRepositorySubPanel( QW.QWidget ):
def _FetchServiceInfo( self ):
service = self._service
def work_callable():
result = service.Request( HC.GET, 'service_info' )
return dict( result[ 'service_info' ] )
def publish_callable( service_info_dict ):
if self._service.GetServiceType() == HC.TAG_REPOSITORY:
l = HC.TAG_REPOSITORY_SERVICE_INFO_TYPES
else:
l = HC.FILE_REPOSITORY_SERVICE_INFO_TYPES
message = 'Note that num file hashes and tags here include deleted content so will likely not line up with your review services value, which is only for current content.'
message += os.linesep * 2
message += os.linesep.join( ( '{}: {}'.format( HC.service_info_enum_str_lookup[ int( info_type ) ], HydrusData.ToHumanInt( info ) ) for ( info_type, info ) in service_info_dict.items() ) )
QW.QMessageBox.information( self, 'Service Info', message )
self._my_updater.Update()
def errback_callable( etype, value, tb ):
if not isinstance( etype, HydrusExceptions.ServerBusyException ):
HydrusData.ShowExceptionTuple( etype, value, tb, do_wait = False )
QW.QMessageBox.critical( self, 'Error', str( value ) )
self._my_updater.Update()
self._service_info_button.setEnabled( False )
self._service_info_button.setText( 'fetching\u2026' )
job = ClientGUIAsync.AsyncQtJob( self, work_callable, publish_callable, errback_callable = errback_callable )
job.start()
def _PausePlayUpdateDownloading( self ):
self._service.PausePlayUpdateDownloading()
@ -2907,6 +2962,13 @@ class ReviewServiceRepositorySubPanel( QW.QWidget ):
#
self._service_info_button.setText( 'service info' )
self._service_info_button.setEnabled( True )
self._service_info_button.setVisible( HG.client_controller.new_options.GetBoolean( 'advanced_mode' ) )
#
all_processing_paused = self._service.IsPausedUpdateProcessing()
if all_processing_paused:
@ -3803,7 +3865,7 @@ class ReviewServiceRatingSubPanel( ClientGUICommon.StaticBox ):
service_info = HG.client_controller.Read( 'service_info', service.GetServiceKey() )
num_files = service_info[ HC.SERVICE_INFO_NUM_FILES ]
num_files = service_info[ HC.SERVICE_INFO_NUM_FILE_HASHES ]
text = HydrusData.ToHumanInt( num_files ) + ' files are rated'
@ -3877,7 +3939,7 @@ class ReviewServiceTagSubPanel( ClientGUICommon.StaticBox ):
service_info = HG.client_controller.Read( 'service_info', service.GetServiceKey() )
num_files = service_info[ HC.SERVICE_INFO_NUM_FILES ]
num_files = service_info[ HC.SERVICE_INFO_NUM_FILE_HASHES ]
num_tags = service_info[ HC.SERVICE_INFO_NUM_TAGS ]
num_mappings = service_info[ HC.SERVICE_INFO_NUM_MAPPINGS ]

93
hydrus/core/HydrusConstants.py
View File

@ -80,7 +80,7 @@ options = {}
# Misc
NETWORK_VERSION = 20
SOFTWARE_VERSION = 497
SOFTWARE_VERSION = 498
CLIENT_API_VERSION = 32
SERVER_THUMBNAIL_DIMENSIONS = ( 200, 200 )
@ -474,14 +474,101 @@ SERVICE_INFO_NUM_UNREAD = 13
SERVICE_INFO_NUM_DRAFTS = 14
SERVICE_INFO_NUM_PENDING_MAPPINGS = 15
SERVICE_INFO_NUM_PETITIONED_MAPPINGS = 16
SERVICE_INFO_NUM_PENDING_FILES = 15
SERVICE_INFO_NUM_PETITIONED_FILES = 16
SERVICE_INFO_NUM_PENDING_TAG_SIBLINGS = 17
SERVICE_INFO_NUM_PETITIONED_TAG_SIBLINGS = 18
SERVICE_INFO_NUM_PENDING_TAG_PARENTS = 19
SERVICE_INFO_NUM_PETITIONED_TAG_PARENTS = 20
SERVICE_INFO_NUM_SHARES = 21
SERVICE_INFO_NUM_VIEWABLE_FILES = 22
SERVICE_INFO_NUM_TAG_SIBLINGS = 23
SERVICE_INFO_NUM_TAG_PARENTS = 24
SERVICE_INFO_NUM_DELETED_TAG_SIBLINGS = 25
SERVICE_INFO_NUM_DELETED_TAG_PARENTS = 26
SERVICE_INFO_NUM_ACTIONABLE_FILE_ADD_PETITIONS = 27
SERVICE_INFO_NUM_ACTIONABLE_FILE_DELETE_PETITIONS = 28
SERVICE_INFO_NUM_ACTIONABLE_MAPPING_ADD_PETITIONS = 29
SERVICE_INFO_NUM_ACTIONABLE_MAPPING_DELETE_PETITIONS = 30
SERVICE_INFO_NUM_ACTIONABLE_SIBLING_ADD_PETITIONS = 31
SERVICE_INFO_NUM_ACTIONABLE_SIBLING_DELETE_PETITIONS = 32
SERVICE_INFO_NUM_ACTIONABLE_PARENT_ADD_PETITIONS = 33
SERVICE_INFO_NUM_ACTIONABLE_PARENT_DELETE_PETITIONS = 34
SERVICE_INFO_NUM_FILE_HASHES = 35
SERVICE_INFO_NUM_PENDING_FILES = 36
SERVICE_INFO_NUM_PETITIONED_FILES = 37
service_info_enum_str_lookup = {
SERVICE_INFO_NUM_FILE_HASHES : 'number of file hashes',
SERVICE_INFO_NUM_FILES : 'number of files',
SERVICE_INFO_NUM_INBOX : 'number in inbox',
SERVICE_INFO_NUM_LOCAL : 'number of local files',
SERVICE_INFO_NUM_MAPPINGS : 'number of mappings',
SERVICE_INFO_NUM_DELETED_MAPPINGS : 'number of deleted mappings',
SERVICE_INFO_NUM_DELETED_FILES : 'number of deleted files',
SERVICE_INFO_NUM_THUMBNAILS : 'number of thumbnails',
SERVICE_INFO_NUM_THUMBNAILS_LOCAL : 'number of local thumbnails',
SERVICE_INFO_TOTAL_SIZE : 'total size',
SERVICE_INFO_NUM_NAMESPACES : 'number of namespaces',
SERVICE_INFO_NUM_TAGS : 'number of tags',
SERVICE_INFO_NUM_PENDING : 'number pending',
SERVICE_INFO_NUM_CONVERSATIONS : 'number of conversations',
SERVICE_INFO_NUM_UNREAD : 'number of unread conversations',
SERVICE_INFO_NUM_DRAFTS : 'number of drafts',
SERVICE_INFO_NUM_PENDING_MAPPINGS : 'number of pending mappings',
SERVICE_INFO_NUM_PETITIONED_MAPPINGS : 'number of petitioned mappings',
SERVICE_INFO_NUM_PENDING_FILES : 'number of pending files',
SERVICE_INFO_NUM_PETITIONED_FILES : 'number of petitioned files',
SERVICE_INFO_NUM_PENDING_TAG_SIBLINGS : 'number of pending tag siblings',
SERVICE_INFO_NUM_PETITIONED_TAG_SIBLINGS : 'number of petitioned tag siblings',
SERVICE_INFO_NUM_PENDING_TAG_PARENTS : 'number of pending tag parents',
SERVICE_INFO_NUM_PETITIONED_TAG_PARENTS : 'number of petitioned tag parents',
SERVICE_INFO_NUM_SHARES : 'number of shares',
SERVICE_INFO_NUM_VIEWABLE_FILES : 'number of viewable files',
SERVICE_INFO_NUM_TAG_SIBLINGS : 'number of tag siblings',
SERVICE_INFO_NUM_TAG_PARENTS : 'number of tag parents',
SERVICE_INFO_NUM_DELETED_TAG_SIBLINGS : 'number of deleted tag siblings',
SERVICE_INFO_NUM_DELETED_TAG_PARENTS : 'number of deleted tag parents',
SERVICE_INFO_NUM_ACTIONABLE_FILE_ADD_PETITIONS : 'number of actionable add-file petitions',
SERVICE_INFO_NUM_ACTIONABLE_FILE_DELETE_PETITIONS : 'number of actionable delete-file petitions',
SERVICE_INFO_NUM_ACTIONABLE_MAPPING_ADD_PETITIONS : 'number of actionable add-mapping petitions',
SERVICE_INFO_NUM_ACTIONABLE_MAPPING_DELETE_PETITIONS : 'number of actionable delete-mapping petitions',
SERVICE_INFO_NUM_ACTIONABLE_SIBLING_ADD_PETITIONS : 'number of actionable add-sibling petitions',
SERVICE_INFO_NUM_ACTIONABLE_SIBLING_DELETE_PETITIONS : 'number of actionable delete-sibling petitions',
SERVICE_INFO_NUM_ACTIONABLE_PARENT_ADD_PETITIONS : 'number of actionable add-parent petitions',
SERVICE_INFO_NUM_ACTIONABLE_PARENT_DELETE_PETITIONS : 'number of actionable delete-parent petitions'
}
FILE_REPOSITORY_SERVICE_INFO_TYPES = [
SERVICE_INFO_NUM_FILE_HASHES,
SERVICE_INFO_NUM_FILES,
SERVICE_INFO_NUM_DELETED_FILES,
SERVICE_INFO_NUM_PENDING_FILES,
SERVICE_INFO_NUM_PETITIONED_FILES,
SERVICE_INFO_NUM_ACTIONABLE_FILE_ADD_PETITIONS,
SERVICE_INFO_NUM_ACTIONABLE_FILE_DELETE_PETITIONS,
]
TAG_REPOSITORY_SERVICE_INFO_TYPES = [
SERVICE_INFO_NUM_FILE_HASHES,
SERVICE_INFO_NUM_TAGS,
SERVICE_INFO_NUM_MAPPINGS,
SERVICE_INFO_NUM_DELETED_MAPPINGS,
SERVICE_INFO_NUM_PENDING_MAPPINGS,
SERVICE_INFO_NUM_PETITIONED_MAPPINGS,
SERVICE_INFO_NUM_ACTIONABLE_MAPPING_ADD_PETITIONS,
SERVICE_INFO_NUM_ACTIONABLE_MAPPING_DELETE_PETITIONS,
SERVICE_INFO_NUM_TAG_SIBLINGS,
SERVICE_INFO_NUM_DELETED_TAG_SIBLINGS,
SERVICE_INFO_NUM_PENDING_TAG_SIBLINGS,
SERVICE_INFO_NUM_PETITIONED_TAG_SIBLINGS,
SERVICE_INFO_NUM_ACTIONABLE_SIBLING_ADD_PETITIONS,
SERVICE_INFO_NUM_ACTIONABLE_SIBLING_DELETE_PETITIONS,
SERVICE_INFO_NUM_TAG_PARENTS,
SERVICE_INFO_NUM_DELETED_TAG_PARENTS,
SERVICE_INFO_NUM_PENDING_TAG_PARENTS,
SERVICE_INFO_NUM_PETITIONED_TAG_PARENTS,
SERVICE_INFO_NUM_ACTIONABLE_PARENT_ADD_PETITIONS,
SERVICE_INFO_NUM_ACTIONABLE_PARENT_DELETE_PETITIONS
]
SERVICE_UPDATE_DELETE_PENDING = 0
SERVICE_UPDATE_RESET = 1

133
hydrus/core/networking/HydrusNetwork.py
View File

@ -2286,67 +2286,106 @@ class Petition( HydrusSerialisable.SerialisableBase ):
SERIALISABLE_TYPE = HydrusSerialisable.SERIALISABLE_TYPE_PETITION
SERIALISABLE_NAME = 'Petition'
SERIALISABLE_VERSION = 1
SERIALISABLE_VERSION = 2
def __init__( self, action = None, petitioner_account = None, reason = None, contents = None ):
def __init__( self, petitioner_account = None, reason = None, actions_and_contents = None ):
if actions_and_contents is None:
actions_and_contents = []
HydrusSerialisable.SerialisableBase.__init__( self )
self._action = action
self._petitioner_account = petitioner_account
self._reason = reason
self._contents = contents
self._actions_and_contents = [ ( action, HydrusSerialisable.SerialisableList( contents ) ) for ( action, contents ) in actions_and_contents ]
def _GetSerialisableInfo( self ):
serialisable_petitioner_account = Account.GenerateSerialisableTupleFromAccount( self._petitioner_account )
serialisable_contents = [ content.GetSerialisableTuple() for content in self._contents ]
serialisable_actions_and_contents = [ ( action, contents.GetSerialisableTuple() ) for ( action, contents ) in self._actions_and_contents ]
return ( self._action, serialisable_petitioner_account, self._reason, serialisable_contents )
return ( serialisable_petitioner_account, self._reason, serialisable_actions_and_contents )
def _InitialiseFromSerialisableInfo( self, serialisable_info ):
( self._action, serialisable_petitioner_account, self._reason, serialisable_contents ) = serialisable_info
( serialisable_petitioner_account, self._reason, serialisable_actions_and_contents ) = serialisable_info
self._petitioner_account = Account.GenerateAccountFromSerialisableTuple( serialisable_petitioner_account )
self._contents = [ HydrusSerialisable.CreateFromSerialisableTuple( serialisable_content ) for serialisable_content in serialisable_contents ]
self._actions_and_contents = [ ( action, HydrusSerialisable.CreateFromSerialisableTuple( serialisable_contents ) ) for ( action, serialisable_contents ) in serialisable_actions_and_contents ]
def GetActionTextAndAction( self ):
def _UpdateSerialisableInfo( self, version, old_serialisable_info ):
action_text = ''
if self._action == HC.CONTENT_UPDATE_PEND:
if version == 1:
action_text += 'ADD'
( action, serialisable_petitioner_account, reason, serialisable_contents ) = old_serialisable_info
elif self._action == HC.CONTENT_UPDATE_PETITION:
contents = [ HydrusSerialisable.CreateFromSerialisableTuple( serialisable_content ) for serialisable_content in serialisable_contents ]
action_text += 'DELETE'
actions_and_contents = [ ( action, HydrusSerialisable.SerialisableList( contents ) ) ]
serialisable_actions_and_contents = [ ( action, contents.GetSerialisableTuple() ) for ( action, contents ) in actions_and_contents ]
new_serialisable_info = ( serialisable_petitioner_account, reason, serialisable_actions_and_contents )
return ( 2, new_serialisable_info )
return ( action_text, self._action )
def GetContents( self, action ):
actions_to_contents = dict( self._actions_and_contents )
if action in actions_to_contents:
return actions_to_contents[ action ]
else:
return []
def GetApproval( self, contents ):
def GetActionsAndContents( self ):
if self._action == HC.CONTENT_UPDATE_PEND:
content_update_action = HC.CONTENT_UPDATE_ADD
elif self._action == HC.CONTENT_UPDATE_PETITION:
content_update_action = HC.CONTENT_UPDATE_DELETE
return self._actions_and_contents
def GetPetitionerAccount( self ):
return self._petitioner_account
def GetReason( self ):
return self._reason
@staticmethod
def GetApproval( action, contents, reason ):
update = ClientToServerUpdate()
content_updates = []
if action == HC.CONTENT_UPDATE_PEND:
content_update_action = HC.CONTENT_UPDATE_ADD
elif action == HC.CONTENT_UPDATE_PETITION:
content_update_action = HC.CONTENT_UPDATE_DELETE
else:
raise Exception( 'Petition came with unexpected action: {}'.format( action ) )
for content in contents:
update.AddContent( self._action, content, self._reason )
update.AddContent( action, content, reason )
content_type = content.GetContentType()
@ -2360,42 +2399,32 @@ class Petition( HydrusSerialisable.SerialisableBase ):
return ( update, content_updates )
def GetContents( self ):
return self._contents
def GetDenial( self, contents ):
if self._action == HC.CONTENT_UPDATE_PEND:
denial_action = HC.CONTENT_UPDATE_DENY_PEND
elif self._action == HC.CONTENT_UPDATE_PETITION:
denial_action = HC.CONTENT_UPDATE_DENY_PETITION
@staticmethod
def GetDenial( action, contents, reason ):
update = ClientToServerUpdate()
if action == HC.CONTENT_UPDATE_PEND:
denial_action = HC.CONTENT_UPDATE_DENY_PEND
elif action == HC.CONTENT_UPDATE_PETITION:
denial_action = HC.CONTENT_UPDATE_DENY_PETITION
else:
raise Exception( 'Petition came with unexpected action: {}'.format( action ) )
for content in contents:
update.AddContent( denial_action, content, self._reason )
update.AddContent( denial_action, content, reason )
return update
def GetPetitionerAccount( self ):
return self._petitioner_account
def GetReason( self ):
return self._reason
HydrusSerialisable.SERIALISABLE_TYPES_TO_OBJECT_TYPES[ HydrusSerialisable.SERIALISABLE_TYPE_PETITION ] = Petition
class ServerService( object ):

1019
hydrus/server/ServerDB.py
View File

File diff suppressed because it is too large Load Diff

6
hydrus/server/networking/ServerServer.py
View File

@ -8,6 +8,8 @@ class HydrusServiceRestricted( HydrusServer.HydrusService ):
root = HydrusServer.HydrusService._InitRoot( self )
root.putChild( b'busy', ServerServerResources.HydrusResourceBusyCheck() )
root.putChild( b'access_key', ServerServerResources.HydrusResourceAccessKey( self._service, HydrusServer.REMOTE_DOMAIN ) )
root.putChild( b'access_key_verification', ServerServerResources.HydrusResourceAccessKeyVerification( self._service, HydrusServer.REMOTE_DOMAIN ) )
root.putChild( b'auto_create_account_types', ServerServerResources.HydrusResourceAutoCreateAccountTypes( self._service, HydrusServer.REMOTE_DOMAIN ) )
@ -35,6 +37,9 @@ class HydrusServiceRestricted( HydrusServer.HydrusService ):
root.putChild( b'registration_keys', ServerServerResources.HydrusResourceRestrictedRegistrationKeys( self._service, HydrusServer.REMOTE_DOMAIN ) )
root.putChild( b'service_info', ServerServerResources.HydrusResourceRestrictedServiceInfo( self._service, HydrusServer.REMOTE_DOMAIN ) )
root.putChild( b'maintenance_regen_service_info', ServerServerResources.HydrusResourceRestrictedMaintenanceRegenServiceInfo( self._service, HydrusServer.REMOTE_DOMAIN ) )
return root
@ -44,7 +49,6 @@ class HydrusServiceAdmin( HydrusServiceRestricted ):
root = HydrusServiceRestricted._InitRoot( self )
root.putChild( b'busy', ServerServerResources.HydrusResourceBusyCheck() )
root.putChild( b'backup', ServerServerResources.HydrusResourceRestrictedBackup( self._service, HydrusServer.REMOTE_DOMAIN ) )
root.putChild( b'lock_on', ServerServerResources.HydrusResourceRestrictedLockOn( self._service, HydrusServer.REMOTE_DOMAIN ) )
root.putChild( b'lock_off', ServerServerResources.HydrusResourceRestrictedLockOff( self._service, HydrusServer.REMOTE_DOMAIN ) )

36
hydrus/server/networking/ServerServerResources.py
View File

@ -879,6 +879,22 @@ class HydrusResourceRestrictedLockOff( HydrusResourceRestricted ):
return response_context
class HydrusResourceRestrictedMaintenanceRegenServiceInfo( HydrusResourceRestricted ):
def _checkAccountPermissions( self, request: HydrusServerRequest.HydrusRequest ):
request.hydrus_account.CheckPermission( HC.CONTENT_TYPE_OPTIONS, HC.PERMISSION_ACTION_MODERATE )
def _threadDoPOSTJob( self, request: HydrusServerRequest.HydrusRequest ):
HG.server_controller.WriteSynchronous( 'maintenance_regen_service_info', self._service_key )
response_context = HydrusServerResources.ResponseContext( 200 )
return response_context
class HydrusResourceRestrictedNumPetitions( HydrusResourceRestricted ):
def _checkAccountPermissions( self, request: HydrusServerRequest.HydrusRequest ):
@ -1077,6 +1093,26 @@ class HydrusResourceRestrictedRepositoryThumbnail( HydrusResourceRestricted ):
return response_context
class HydrusResourceRestrictedServiceInfo( HydrusResourceRestricted ):
def _checkAccountPermissions( self, request: HydrusServerRequest.HydrusRequest ):
# you can always fetch the service info
pass
def _threadDoGETJob( self, request: HydrusServerRequest.HydrusRequest ):
service_info = HG.server_controller.Read( 'service_info', self._service_key )
body = HydrusNetworkVariableHandling.DumpHydrusArgsToNetworkBytes( { 'service_info' : list( service_info.items() ) } )
response_context = HydrusServerResources.ResponseContext( 200, body = body )
return response_context
class HydrusResourceRestrictedServices( HydrusResourceRestricted ):
def _checkAccountPermissions( self, request: HydrusServerRequest.HydrusRequest ):

5
hydrus/test/TestHydrusServer.py
View File

@ -383,12 +383,11 @@ class TestServer( unittest.TestCase ):
# petition
action = HC.CONTENT_UPDATE_PETITION
petitioner_account = HydrusNetwork.Account.GenerateUnknownAccount()
reason = 'it sucks'
contents = [ HydrusNetwork.Content( HC.CONTENT_TYPE_FILES, [ HydrusData.GenerateKey() for i in range( 10 ) ] ) ]
actions_and_contents = ( HC.CONTENT_UPDATE_PETITION, [ HydrusNetwork.Content( HC.CONTENT_TYPE_FILES, [ HydrusData.GenerateKey() for i in range( 10 ) ] ) ] )
petition = HydrusNetwork.Petition( action, petitioner_account, reason, contents )
petition = HydrusNetwork.Petition( petitioner_account, reason, actions_and_contents )
HG.test_controller.SetRead( 'petition', petition )

615
hydrus/test/TestServerDB.py
View File

@ -1,3 +1,5 @@
import os
import random
import time
import unittest
@ -47,7 +49,7 @@ class TestServerDB( unittest.TestCase ):
#
self._regular_user_account_type = HydrusNetwork.AccountType.GenerateNewAccountType( 'regular user', { HC.CONTENT_TYPE_MAPPINGS : HC.PERMISSION_ACTION_CREATE }, HydrusNetworking.BandwidthRules() )
self._regular_user_account_type = HydrusNetwork.AccountType.GenerateNewAccountType( 'regular user', { HC.CONTENT_TYPE_MAPPINGS : HC.PERMISSION_ACTION_CREATE, HC.CONTENT_TYPE_TAG_PARENTS : HC.PERMISSION_ACTION_PETITION, HC.CONTENT_TYPE_TAG_SIBLINGS : HC.PERMISSION_ACTION_PETITION }, HydrusNetworking.BandwidthRules() )
self._deletee_user_account_type = HydrusNetwork.AccountType.GenerateNewAccountType( 'deletee user', {}, HydrusNetworking.BandwidthRules() )
new_account_types = [ self._tag_service_admin_account_type, self._null_account_type, self._regular_user_account_type, self._deletee_user_account_type ]
@ -125,6 +127,28 @@ class TestServerDB( unittest.TestCase ):
def _test_account_fetching_from_content( self ):
tag = 'character:samus aran'
hash = HydrusData.GenerateKey()
mappings_content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( tag, ( hash, ) ) )
mapping_content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPING, ( tag, hash ) )
client_to_server_update = HydrusNetwork.ClientToServerUpdate()
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, mappings_content )
self._write( 'update', self._tag_service_key, self._tag_service_regular_account, client_to_server_update, HydrusData.GetNow() )
# can extend this to generate and fetch an actual update given a timespan
#
result = self._read( 'account_from_content', self._tag_service_key, mapping_content )
self.assertEqual( result.GetAccountKey(), self._tag_service_regular_account.GetAccountKey() )
def _test_account_modification( self ):
regular_account_key = self._tag_service_regular_account.GetAccountKey()
@ -241,27 +265,587 @@ class TestServerDB( unittest.TestCase ):
self.assertEqual( message, set_message )
def _test_content_creation( self ):
def _test_content_creation_and_service_info_counts_files( self ):
tag = 'character:samus aran'
hash = HydrusData.GenerateKey()
# files
mappings_content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( tag, ( hash, ) ) )
mapping_content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPING, ( tag, hash ) )
hashes = [ HydrusData.GenerateKey() for i in range( 20 ) ]
service_info = self._read( 'service_info', self._file_service_key )
expected_num_files = service_info[ HC.SERVICE_INFO_NUM_FILES ]
expected_num_hashes = service_info[ HC.SERVICE_INFO_NUM_FILE_HASHES ]
expected_num_deleted_files = service_info[ HC.SERVICE_INFO_NUM_DELETED_FILES ]
expected_num_petitioned_files = service_info[ HC.SERVICE_INFO_NUM_PETITIONED_FILES ]
expected_num_actionable_delete_petitions = service_info[ HC.SERVICE_INFO_NUM_ACTIONABLE_FILE_DELETE_PETITIONS ]
def do_num_test():
service_info = self._read( 'service_info', self._file_service_key )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_FILES ], expected_num_files )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_FILE_HASHES ], expected_num_hashes )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_DELETED_FILES ], expected_num_deleted_files )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_PETITIONED_FILES ], expected_num_petitioned_files )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_ACTIONABLE_FILE_DELETE_PETITIONS ], expected_num_actionable_delete_petitions )
# add with admin
for ( i, hash ) in enumerate( hashes ):
file_dict = {
'hash' : hash,
'ip' : '127.0.0.1',
'height' : 200,
'width' : 200,
'mime' : 2,
'size' : 5270,
'path' : os.path.join( HC.STATIC_DIR, 'hydrus.png' ),
'thumbnail' : b'abcd'
}
self._write( 'file', self._file_service, self._file_service_account, file_dict, HydrusData.GetNow() )
expected_num_files += 1
expected_num_hashes += 1
do_num_test()
# delete with admin
NUM_TO_DELETE = 5
deletee_hashes = random.sample( hashes, NUM_TO_DELETE )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_FILES, deletee_hashes )
client_to_server_update = HydrusNetwork.ClientToServerUpdate()
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PETITION, content, reason = 'files are bad' )
self._write( 'update', self._file_service_key, self._file_service_account, client_to_server_update, HydrusData.GetNow() )
expected_num_files -= NUM_TO_DELETE
expected_num_deleted_files += NUM_TO_DELETE
do_num_test()
# can't do this until I have a regular account set up earlier in this big test
# petition with regular
# approve and deny some with admin
def _test_content_creation_and_service_info_counts_mappings( self ):
service_info = self._read( 'service_info', self._tag_service_key )
expected_num_tags = service_info[ HC.SERVICE_INFO_NUM_TAGS ]
expected_num_hashes = service_info[ HC.SERVICE_INFO_NUM_FILE_HASHES ]
expected_num_mappings = service_info[ HC.SERVICE_INFO_NUM_MAPPINGS ]
expected_num_deleted_mappings = service_info[ HC.SERVICE_INFO_NUM_DELETED_MAPPINGS ]
expected_num_petitioned_mappings = service_info[ HC.SERVICE_INFO_NUM_PETITIONED_MAPPINGS ]
expected_num_actionable_delete_petitions = service_info[ HC.SERVICE_INFO_NUM_ACTIONABLE_MAPPING_DELETE_PETITIONS ]
def do_num_test():
service_info = self._read( 'service_info', self._tag_service_key )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_TAGS ], expected_num_tags )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_FILE_HASHES ], expected_num_hashes )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_MAPPINGS ], expected_num_mappings )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_DELETED_MAPPINGS ], expected_num_deleted_mappings )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_PETITIONED_MAPPINGS ], expected_num_petitioned_mappings )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_ACTIONABLE_MAPPING_DELETE_PETITIONS ], expected_num_actionable_delete_petitions )
tag_1 = 'character:samus aran'
tag_2 = 'typo'
tag_1_hashes = [ HydrusData.GenerateKey() for i in range( 100 ) ]
tag_2_hashes = tag_1_hashes[50:] + [ HydrusData.GenerateKey() for i in range( 50 ) ]
# add with admin
client_to_server_update = HydrusNetwork.ClientToServerUpdate()
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, mappings_content )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( tag_1, tag_1_hashes ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( tag_2, tag_2_hashes ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content )
self._write( 'update', self._tag_service_key, self._tag_service_account, client_to_server_update, HydrusData.GetNow() )
expected_num_tags += 2
expected_num_hashes += len( set( tag_1_hashes ).union( tag_2_hashes ) )
expected_num_mappings += len( tag_1_hashes ) + len( tag_2_hashes )
do_num_test()
# delete with admin
tag_1_deletee_hashes = random.sample( tag_1_hashes, 15 )
tag_2_deletee_hashes = random.sample( tag_2_hashes, 25 )
client_to_server_update = HydrusNetwork.ClientToServerUpdate()
content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( tag_1, tag_1_deletee_hashes ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PETITION, content, reason = 'admin delete' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( tag_2, tag_2_deletee_hashes ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PETITION, content, reason = 'admin delete' )
self._write( 'update', self._tag_service_key, self._tag_service_account, client_to_server_update, HydrusData.GetNow() )
expected_num_mappings -= len( tag_1_deletee_hashes ) + len( tag_2_deletee_hashes )
expected_num_deleted_mappings += len( tag_1_deletee_hashes ) + len( tag_2_deletee_hashes )
do_num_test()
# petition with regular
tag_1_petition_hashes = random.sample( set( tag_1_hashes ).difference( tag_1_deletee_hashes ), 15 )
tag_2_petition_hashes = random.sample( set( tag_2_hashes ).difference( tag_2_deletee_hashes ), 25 )
tag_2a_petition_hashes = random.sample( set( tag_2_hashes ).difference( tag_2_deletee_hashes ).difference( tag_2_petition_hashes ), 10 )
client_to_server_update = HydrusNetwork.ClientToServerUpdate()
content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( tag_1, tag_1_petition_hashes ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PETITION, content, reason = 'these are bad' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( tag_2, tag_2_petition_hashes ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PETITION, content, reason = 'these are bad' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( tag_2, tag_2a_petition_hashes ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PETITION, content, reason = 'bad for a stupid reason' )
self._write( 'update', self._tag_service_key, self._tag_service_regular_account, client_to_server_update, HydrusData.GetNow() )
# can extend this to generate and fetch an actual update given a timespan
expected_num_petitioned_mappings += len( tag_1_petition_hashes ) + len( tag_2_petition_hashes ) + len( tag_2a_petition_hashes )
expected_num_actionable_delete_petitions += 3
#
do_num_test()
result = self._read( 'account_from_content', self._tag_service_key, mapping_content )
# approve and deny some with admin
self.assertEqual( result.GetAccountKey(), self._tag_service_regular_account.GetAccountKey() )
client_to_server_update = HydrusNetwork.ClientToServerUpdate()
content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( tag_1, tag_1_petition_hashes ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PETITION, content, reason = 'these are bad' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( tag_2, tag_2_petition_hashes ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PETITION, content, reason = 'these are bad' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( tag_2, tag_2a_petition_hashes ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_DENY_PETITION, content )
self._write( 'update', self._tag_service_key, self._tag_service_account, client_to_server_update, HydrusData.GetNow() )
expected_num_mappings -= len( tag_1_petition_hashes ) + len( tag_2_petition_hashes )
expected_num_deleted_mappings += len( tag_1_petition_hashes ) + len( tag_2_petition_hashes )
expected_num_petitioned_mappings -= len( tag_1_petition_hashes ) + len( tag_2_petition_hashes ) + len( tag_2a_petition_hashes )
expected_num_actionable_delete_petitions -= 3
do_num_test()
def _test_content_creation_and_service_info_counts_tag_parents( self ):
service_info = self._read( 'service_info', self._tag_service_key )
expected_num_tags = service_info[ HC.SERVICE_INFO_NUM_TAGS ]
expected_num_tag_parents = service_info[ HC.SERVICE_INFO_NUM_TAG_PARENTS ]
expected_num_deleted_tag_parents = service_info[ HC.SERVICE_INFO_NUM_DELETED_TAG_PARENTS ]
expected_num_pending_tag_parents = service_info[ HC.SERVICE_INFO_NUM_PENDING_TAG_PARENTS ]
expected_num_actionable_add_petitions = service_info[ HC.SERVICE_INFO_NUM_ACTIONABLE_PARENT_ADD_PETITIONS ]
expected_num_petitioned_tag_parents = service_info[ HC.SERVICE_INFO_NUM_PETITIONED_TAG_PARENTS ]
expected_num_actionable_delete_petitions = service_info[ HC.SERVICE_INFO_NUM_ACTIONABLE_PARENT_DELETE_PETITIONS ]
def do_num_test():
service_info = self._read( 'service_info', self._tag_service_key )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_TAGS ], expected_num_tags )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_TAG_PARENTS ], expected_num_tag_parents )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_DELETED_TAG_PARENTS ], expected_num_deleted_tag_parents )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_PENDING_TAG_PARENTS ], expected_num_pending_tag_parents )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_ACTIONABLE_PARENT_ADD_PETITIONS ], expected_num_actionable_add_petitions )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_PETITIONED_TAG_PARENTS ], expected_num_petitioned_tag_parents )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_ACTIONABLE_PARENT_DELETE_PETITIONS ], expected_num_actionable_delete_petitions )
child_tag_1 = 'child_tag_1'
parent_tag_1 = 'parent_tag_1'
child_tag_2 = 'child_tag_2'
parent_tag_2 = 'parent_tag_2'
child_tag_2a = 'child_tag_2a'
parent_tag_2a = 'parent_tag_2a'
child_tag_3 = 'child_tag_3'
parent_tag_3 = 'parent_tag_3'
replacement_parent_tag_3 = 'replacement_parent_tag_3'
child_tag_4 = 'child_tag_4'
parent_tag_4 = 'parent_tag_4'
replacement_parent_tag_4 = 'replacement_parent_tag_4'
# get some into the db as existing tags, since count adjustment logic can change
hashes = [ HydrusData.GenerateKey() for i in range( 5 ) ]
client_to_server_update = HydrusNetwork.ClientToServerUpdate()
content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( child_tag_2, hashes ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( parent_tag_2, hashes ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( child_tag_4, hashes ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( parent_tag_4, hashes ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( replacement_parent_tag_4, hashes ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content )
self._write( 'update', self._tag_service_key, self._tag_service_account, client_to_server_update, HydrusData.GetNow() )
service_info = self._read( 'service_info', self._tag_service_key )
expected_num_tags += 5
do_num_test()
# add with admin
client_to_server_update = HydrusNetwork.ClientToServerUpdate()
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_PARENTS, ( child_tag_1, parent_tag_1 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content, reason = 'admin add' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_PARENTS, ( child_tag_3, parent_tag_3 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content, reason = 'admin add' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_PARENTS, ( child_tag_4, parent_tag_4 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content, reason = 'admin add' )
self._write( 'update', self._tag_service_key, self._tag_service_account, client_to_server_update, HydrusData.GetNow() )
expected_num_tags += 4
expected_num_tag_parents += 3
do_num_test()
# delete with admin
client_to_server_update = HydrusNetwork.ClientToServerUpdate()
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_PARENTS, ( child_tag_1, parent_tag_1 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PETITION, content, reason = 'admin delete' )
self._write( 'update', self._tag_service_key, self._tag_service_account, client_to_server_update, HydrusData.GetNow() )
expected_num_tag_parents -= 1
expected_num_deleted_tag_parents += 1
do_num_test()
# pend with regular
client_to_server_update = HydrusNetwork.ClientToServerUpdate()
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_PARENTS, ( child_tag_2, parent_tag_2 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content, reason = 'this is great' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_PARENTS, ( child_tag_2a, parent_tag_2a ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content, reason = 'this is great' )
self._write( 'update', self._tag_service_key, self._tag_service_regular_account, client_to_server_update, HydrusData.GetNow() )
expected_num_pending_tag_parents += 2
expected_num_actionable_add_petitions += 1
do_num_test()
# approve and deny some with admin
client_to_server_update = HydrusNetwork.ClientToServerUpdate()
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_PARENTS, ( child_tag_2, parent_tag_2 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content, reason = 'this is great' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_PARENTS, ( child_tag_2a, parent_tag_2a ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_DENY_PEND, content )
self._write( 'update', self._tag_service_key, self._tag_service_account, client_to_server_update, HydrusData.GetNow() )
# note no increase in num tags. _2 already known, _2a denied
expected_num_tag_parents += 1
expected_num_pending_tag_parents -= 2
expected_num_actionable_add_petitions -= 1
do_num_test()
# petition with regular
client_to_server_update = HydrusNetwork.ClientToServerUpdate()
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_PARENTS, ( child_tag_2, parent_tag_2 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PETITION, content, reason = 'this is bad 2' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_PARENTS, ( child_tag_3, parent_tag_3 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PETITION, content, reason = 'replacing 3' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_PARENTS, ( child_tag_4, parent_tag_4 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PETITION, content, reason = 'replacing 4' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_PARENTS, ( child_tag_3, replacement_parent_tag_3 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content, reason = 'replacing 3' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_PARENTS, ( child_tag_4, replacement_parent_tag_4 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content, reason = 'replacing 4' )
self._write( 'update', self._tag_service_key, self._tag_service_regular_account, client_to_server_update, HydrusData.GetNow() )
expected_num_petitioned_tag_parents += 3
expected_num_pending_tag_parents += 2
expected_num_actionable_delete_petitions += 3
do_num_test()
# approve and deny some with admin
client_to_server_update = HydrusNetwork.ClientToServerUpdate()
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_PARENTS, ( child_tag_2, parent_tag_2 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PETITION, content, reason = 'this is bad 2' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_PARENTS, ( child_tag_3, parent_tag_3 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PETITION, content, reason = 'replacing 3' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_PARENTS, ( child_tag_4, parent_tag_4 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_DENY_PETITION, content, reason = 'replacing 4' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_PARENTS, ( child_tag_3, replacement_parent_tag_3 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content, reason = 'replacing 3' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_PARENTS, ( child_tag_4, replacement_parent_tag_4 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_DENY_PEND, content, reason = 'replacing 4' )
self._write( 'update', self._tag_service_key, self._tag_service_account, client_to_server_update, HydrusData.GetNow() )
expected_num_tags += 1
expected_num_tag_parents -= 1
expected_num_deleted_tag_parents += 2
expected_num_petitioned_tag_parents -= 3
expected_num_pending_tag_parents -= 2
expected_num_actionable_delete_petitions -= 3
do_num_test()
def _test_content_creation_and_service_info_counts_tag_siblings( self ):
service_info = self._read( 'service_info', self._tag_service_key )
expected_num_tags = service_info[ HC.SERVICE_INFO_NUM_TAGS ]
expected_num_tag_siblings = service_info[ HC.SERVICE_INFO_NUM_TAG_SIBLINGS ]
expected_num_deleted_tag_siblings = service_info[ HC.SERVICE_INFO_NUM_DELETED_TAG_SIBLINGS ]
expected_num_pending_tag_siblings = service_info[ HC.SERVICE_INFO_NUM_PENDING_TAG_SIBLINGS ]
expected_num_actionable_add_petitions = service_info[ HC.SERVICE_INFO_NUM_ACTIONABLE_SIBLING_ADD_PETITIONS ]
expected_num_petitioned_tag_siblings = service_info[ HC.SERVICE_INFO_NUM_PETITIONED_TAG_SIBLINGS ]
expected_num_actionable_delete_petitions = service_info[ HC.SERVICE_INFO_NUM_ACTIONABLE_SIBLING_DELETE_PETITIONS ]
def do_num_test():
service_info = self._read( 'service_info', self._tag_service_key )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_TAGS ], expected_num_tags )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_TAG_SIBLINGS ], expected_num_tag_siblings )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_DELETED_TAG_SIBLINGS ], expected_num_deleted_tag_siblings )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_PENDING_TAG_SIBLINGS ], expected_num_pending_tag_siblings )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_ACTIONABLE_SIBLING_ADD_PETITIONS ], expected_num_actionable_add_petitions )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_PETITIONED_TAG_SIBLINGS ], expected_num_petitioned_tag_siblings )
self.assertEqual( service_info[ HC.SERVICE_INFO_NUM_ACTIONABLE_SIBLING_DELETE_PETITIONS ], expected_num_actionable_delete_petitions )
bad_tag_1 = 'bad_tag_1'
good_tag_1 = 'good_tag_1'
bad_tag_2 = 'bad_tag_2'
good_tag_2 = 'good_tag_2'
bad_tag_2a = 'bad_tag_2a'
good_tag_2a = 'good_tag_2a'
bad_tag_3 = 'bad_tag_3'
good_tag_3 = 'good_tag_3'
replacement_good_tag_3 = 'replacement_good_tag_3'
bad_tag_4 = 'bad_tag_4'
good_tag_4 = 'good_tag_4'
replacement_good_tag_4 = 'replacement_good_tag_4'
# get some into the db as existing tags, since count adjustment logic can change
hashes = [ HydrusData.GenerateKey() for i in range( 5 ) ]
client_to_server_update = HydrusNetwork.ClientToServerUpdate()
content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( bad_tag_2, hashes ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( good_tag_2, hashes ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( bad_tag_4, hashes ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( good_tag_4, hashes ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_MAPPINGS, ( replacement_good_tag_4, hashes ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content )
self._write( 'update', self._tag_service_key, self._tag_service_account, client_to_server_update, HydrusData.GetNow() )
service_info = self._read( 'service_info', self._tag_service_key )
expected_num_tags += 5
do_num_test()
# add with admin
client_to_server_update = HydrusNetwork.ClientToServerUpdate()
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_SIBLINGS, ( bad_tag_1, good_tag_1 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content, reason = 'admin add' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_SIBLINGS, ( bad_tag_3, good_tag_3 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content, reason = 'admin add' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_SIBLINGS, ( bad_tag_4, good_tag_4 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content, reason = 'admin add' )
self._write( 'update', self._tag_service_key, self._tag_service_account, client_to_server_update, HydrusData.GetNow() )
expected_num_tags += 4
expected_num_tag_siblings += 3
do_num_test()
# delete with admin
client_to_server_update = HydrusNetwork.ClientToServerUpdate()
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_SIBLINGS, ( bad_tag_1, good_tag_1 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PETITION, content, reason = 'admin delete' )
self._write( 'update', self._tag_service_key, self._tag_service_account, client_to_server_update, HydrusData.GetNow() )
expected_num_tag_siblings -= 1
expected_num_deleted_tag_siblings += 1
do_num_test()
# pend with regular
client_to_server_update = HydrusNetwork.ClientToServerUpdate()
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_SIBLINGS, ( bad_tag_2, good_tag_2 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content, reason = 'this is great' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_SIBLINGS, ( bad_tag_2a, good_tag_2a ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content, reason = 'this is great' )
self._write( 'update', self._tag_service_key, self._tag_service_regular_account, client_to_server_update, HydrusData.GetNow() )
expected_num_pending_tag_siblings += 2
expected_num_actionable_add_petitions += 1
do_num_test()
# approve and deny some with admin
client_to_server_update = HydrusNetwork.ClientToServerUpdate()
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_SIBLINGS, ( bad_tag_2, good_tag_2 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content, reason = 'this is great' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_SIBLINGS, ( bad_tag_2a, good_tag_2a ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_DENY_PEND, content )
self._write( 'update', self._tag_service_key, self._tag_service_account, client_to_server_update, HydrusData.GetNow() )
# note no increase in num tags. _2 already known, _2a denied
expected_num_tag_siblings += 1
expected_num_pending_tag_siblings -= 2
expected_num_actionable_add_petitions -= 1
do_num_test()
# petition with regular
client_to_server_update = HydrusNetwork.ClientToServerUpdate()
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_SIBLINGS, ( bad_tag_2, good_tag_2 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PETITION, content, reason = 'this is bad 2' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_SIBLINGS, ( bad_tag_3, good_tag_3 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PETITION, content, reason = 'replacing 3' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_SIBLINGS, ( bad_tag_4, good_tag_4 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PETITION, content, reason = 'replacing 4' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_SIBLINGS, ( bad_tag_3, replacement_good_tag_3 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content, reason = 'replacing 3' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_SIBLINGS, ( bad_tag_4, replacement_good_tag_4 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content, reason = 'replacing 4' )
self._write( 'update', self._tag_service_key, self._tag_service_regular_account, client_to_server_update, HydrusData.GetNow() )
expected_num_petitioned_tag_siblings += 3
expected_num_pending_tag_siblings += 2
expected_num_actionable_delete_petitions += 3
do_num_test()
# approve and deny some with admin
client_to_server_update = HydrusNetwork.ClientToServerUpdate()
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_SIBLINGS, ( bad_tag_2, good_tag_2 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PETITION, content, reason = 'this is bad 2' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_SIBLINGS, ( bad_tag_3, good_tag_3 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PETITION, content, reason = 'replacing 3' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_SIBLINGS, ( bad_tag_4, good_tag_4 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_DENY_PETITION, content, reason = 'replacing 4' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_SIBLINGS, ( bad_tag_3, replacement_good_tag_3 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_PEND, content, reason = 'replacing 3' )
content = HydrusNetwork.Content( HC.CONTENT_TYPE_TAG_SIBLINGS, ( bad_tag_4, replacement_good_tag_4 ) )
client_to_server_update.AddContent( HC.CONTENT_UPDATE_DENY_PEND, content, reason = 'replacing 4' )
self._write( 'update', self._tag_service_key, self._tag_service_account, client_to_server_update, HydrusData.GetNow() )
expected_num_tags += 1
expected_num_tag_siblings -= 1
expected_num_deleted_tag_siblings += 2
expected_num_petitioned_tag_siblings -= 3
expected_num_pending_tag_siblings -= 2
expected_num_actionable_delete_petitions -= 3
do_num_test()
def _test_init_server_admin( self ):
@ -331,8 +915,13 @@ class TestServerDB( unittest.TestCase ):
self._test_account_creation()
self._test_content_creation()
self._test_account_modification()
self._test_content_creation_and_service_info_counts_files()
self._test_content_creation_and_service_info_counts_mappings()
self._test_content_creation_and_service_info_counts_tag_parents()
self._test_content_creation_and_service_info_counts_tag_siblings()
self._test_account_fetching_from_content()