Version 535
This commit is contained in:
Hydrus Network Developer
parent
64f946c58f
commit
27e1bb5b85
|
|
@ -7,6 +7,49 @@ title: Changelog
|
|||
!!! note
|
||||
This is the new changelog, only the most recent builds. For all versions, see the [old changelog](old_changelog.html).
|
||||
|
||||
## [Version 535](https://github.com/hydrusnetwork/hydrus/releases/tag/v535)
|
||||
|
||||
### misc
|
||||
|
||||
* thanks to a user, we now have Krita (.kra, .krz) support! it even pulls thumbnails]
|
||||
* thanks to another user, we now have SVG (.svg) support! it even generates thumbnails!
|
||||
* I think I fixed a comparison statement calculator divide-by-zero error in the duplicate filter when you compare a file with a resolution with a file without one
|
||||
|
||||
### petitions overview
|
||||
|
||||
* _this is a workflow/usability update only for server janitors_
|
||||
* tl;dr: the petitions page now fetches many petitions at once. update your servers and clients for it all to work right
|
||||
* so, the petitions page now fetches lots of petitions with each 'fetch' button click. you can set how many it will fetch with a new number control
|
||||
* the petitions are shown in a new multi-column list that shows action, account id, reason, and total weight. the actual data for the petitions will load in quickly, reflected in the list. as soon as the first is loaded, it is highlighted, but double-click any to highlight it in the old petition UI as normal
|
||||
* when you process petitions, the client moves instantly to the next, all fitting into the existing workflow, without having to wait for the server to fetch a new one after you commit
|
||||
* you can also mass approve/deny from here! if one account is doing great or terrible stuff, you can now blang it all in one go
|
||||
|
||||
### petitions details
|
||||
|
||||
* the 'fetch x petition' buttons now show `(*)` in their label if they are the active petition type being worked on
|
||||
* petition pages now remember: the last petition type they were looking at; the number of petitions to fetch; and the number of files to show
|
||||
* the petition page will pause any ongoing petition fetches if you close it, and resume if you unclose it
|
||||
* a system where multi-mapping petitions would be broken up and delivered in tags with weight-similar chunks (e.g. if would say 'aaa for 11 files' and 'bbb in 15 files' in the same fetch, but not 'ccc in 542,154 files') is abandoned. this was not well explained and was causing confusion and code complexity. these petitions now appear clientside in full
|
||||
* another system, where multi-mapping petitions would be delivered in same-namespace chunks, is also abandoned, for similar reasons. it was causing more confusion, especially when compared to the newer petition counting tech I've added. perhaps it will come back in as a clientside filter option
|
||||
* the list of petitions you are given _should_ also be neatly grouped by account id, so rather than randomly sampling from all petitions, you'll get batches by user x, y, or z, and in most cases you'll be looking at everything by user x, and y, and then z up to the limit of num petitions you chose to fetch
|
||||
* drawback: since petitions' content can overlap in complicated ways, and janitors can work on the same list at the same time, in edge cases the list you see can be slightly out of sync with what the server actually has. this isn't a big deal, and the worst case is wasted work as you approve the same thing twice. I tried to implement 'refresh list if count drops more than expected' tech, but the situation is complicated and it was spamming too much. I will let you refresh the list with a button click yourself for now, as you like, and please let me know where it works and fails
|
||||
* drawback: I added some new objects, so you have to update both server and client for this to work. older/newer combinations will give you some harmless errors
|
||||
* also, if your list starts running low, but there are plenty more petitions to work on, it will auto-refresh. again, it won't interrupt your current work, but it will fetch more. let me know how it works out
|
||||
* drawback: while the new petition summary list is intentionally lightweight, I do spend some extra CPU figuring it out. with a high 'num petitions to fetch', it may take several seconds for a very busy server like the PTR just to fetch the initial list, so please play around with different fetch sizes and let me know what works well and what is way too slow
|
||||
* there are still some things I want to do to this page, which I want to slip in the near future. I want to hide/show the sort and 'num files to show' widgets as appropriate, figure out a right-click menu for the new list to retry failures, and get some shortcut support going
|
||||
|
||||
### boring code cleanup
|
||||
|
||||
* wrote a new petition header object to hold content type, petition status, account id, and reason for petitions
|
||||
* serverside petition fetching is now split into 'get petition headers' and 'get petition data'. the 'headers' section supports filtering by account id and in future reason
|
||||
* the clientside petition management UI code pretty much got a full pass
|
||||
* cleaned a bunch of ancient server db code
|
||||
* cleaned a bunch of the clientside petition code. it was a real tangle
|
||||
* improved the resilience of the hydrus server when it is given unacceptable tags in a content update
|
||||
* all fetches of multiple rows of data from multi-column lists now happen sorted. this is just a little thing, but it'll probably dejank a few operations where you edit several things at once or get some errors and are trying to figure out which of five things caused it
|
||||
* the hydrus official mimetype for psd files is now 'image/vnd.adobe.photoshop' (instead of 'application/x-photoshop')
|
||||
* with krita file (which are actually just zip files) support, we now have the very barebones of archive tech started. I'll expand it a bit more and we should be able to improve support for other archive-like formats in the future
|
||||
|
||||
## [Version 534](https://github.com/hydrusnetwork/hydrus/releases/tag/v534)
|
||||
|
||||
### user submissions
|
||||
|
|
@ -332,28 +375,3 @@ title: Changelog
|
|||
* the main build script no longer uses set-output commands (these are deprecated and being dropped later in the year I think, in favour of some ENV stuff)
|
||||
* tidied some cruft from the main build script
|
||||
* I moved the 'new' python-mpv in the requirements.txts from 1.0.1 to 1.0.3. source users might like to rebuild their venvs again, particularly Windows users who updated to the new mpv dll recently
|
||||
|
||||
## [Version 525](https://github.com/hydrusnetwork/hydrus/releases/tag/v525)
|
||||
|
||||
### library updates
|
||||
|
||||
* after successful testing amongst source users, I am finally updating the official builds and the respective requirements.txts for Qt, from 6.3.1 to 6.4.1 (with 'test' now 6.5.0), opencv-python-headless from 4.5.3.56 to 4.5.5.64 (with a new 'test' of 4.7.0.72), and in the Windows build, the mpv dll from 2022-05-01 to 2023-02-12 (API 2.0 to 2.1). if you use my normal builds, you don't have to do anything special in the update, and with luck you'll get slightly faster images, video, and UI, and with fewer bugs. if you run from source, you might want to re-run your setup_venv script--it'll update you automatically--and if you are a modern Windows source user and haven't yet, grab the new dll here and rename it to mpv-2.dll https://sourceforge.net/projects/mpv-player-windows/files/libmpv/mpv-dev-x86_64-20230212-git-a40958c.7z . there is a chance that some older OSes will not be able to boot this new build, but I think these people were already migrated to being source users when Win 7-level was no longer supported. in any case, let me know how you get on, and if you are on an older OS, be prepared to rollback if this version doesn't boot
|
||||
* setup_venv.bat (Windows source) now adds PyWin32, just like the builds (the new version of pympler, a memory management module, moans on boot if it doesn't have it)
|
||||
|
||||
### timestamps
|
||||
|
||||
* a couple places where fixed calendar time-deltas are converted to absolute datestrings now work better over longer times. going back (5 years, 3 months) should now work out the actual calendar dates (previously they used a rough total_num_seconds estimation) and go back to the same day of the destination month, also accounting for if that has fewer days than the starting month and handling leap years. it also handles >'12 months' better now
|
||||
* in system:time predicates that use since/before a delta, it now allows much larger values in the UI, like '72 months', and it won't merge those into the larger values in the label. so if you set a gap of 100 days, it'll say that, not 3 months 10 days or whatever
|
||||
* the main copy button on 'manage file times' is now a menu button letting you choose to copy all timestamps or just those for the file services. as a hacky experiment, you can also copy the file service timestamps plus one second (in case you want to try finick-ily going through a handful of files to force a certain import sort order)
|
||||
* the system predicate time parsing is now more flexible. for archived, modified, last viewed, and imported time, you can now generally say all variants in the form 'import' or 'imported' and 'time' or 'date' and 'time imported' or 'imported time'.
|
||||
* fixed an issue that meant editing existing delta 'system:archived time' predicates was launching the 'date' edit panel
|
||||
|
||||
### misc
|
||||
|
||||
* in the 'exif and other embedded metadata' review window, which is launched from a button on the the media viewer's top hover, jpegs now state their subsampling and whether they are progressive
|
||||
* every simple place where the client eats clipboard data and tries to import something now has a unified error-reporting process. before, it would make a popup with something like 'I could not understandwhat was in the clipboard!'. Now it makes a popup with info on what was pasted, what was expected, and actual exception info. Longer info is printed to the log
|
||||
* many places across the program say the specific exception type when they report errors now, not just the string summary
|
||||
* the sankaku downloader is updated with a new url class for their new md5 links. also, the file parser is updated to associate the old id URL, and the gallery parser is updated to skip the 'get sank pro' thumbnail links if you are not logged in. if you have sank subscriptions, they are going to go crazy this week due to the URL format changing--sorry, there's no nice way around it!--just ignore their popups about hitting file limits and wait them out. unfortunately, due to an unusual 404-based redirect, the id-based URLs will not work in hydrus any more
|
||||
* the 'API URL' system for url classes now supports File URLs--this may help you figure out some CDN redirects and similar. in a special rule for these File URLs, both URLs will be associated with the imported file (normally, Post API URLs are not saved as Known URLs). relatedly, I have renamed this system broadly to 'api/redirect url', since we use it for a bunch of non-API stuff now
|
||||
* fixed a problem where deleting one of the new inc/dec rating services was not clearing the actual number ratings for that service from the database, causing service-id error hell on loading files with those orphaned rating records. sorry for the trouble, this slipped through testing! any users who were affected by this will also be fixed (orphan records cleared out) on update (issue #1357)
|
||||
* the client cleans up the temporary paths used by file imports more carefully now: it tries more times to delete 'sticky' temp files; it tries to clear them again immediately on shutdown; and it stores them all in the hydrus temp subdirectory where they are less loose and will be captured by the final directory clear on shutdown (issue #1356)
|
||||
|
|
|
|||
|
|
@ -72,6 +72,7 @@ Now:
|
|||
* **image/png** (.png)
|
||||
* **image/apng** (.apng)
|
||||
* **image/jpeg** (.jpg)
|
||||
* **image/svg+xml** (.svg)
|
||||
* **image/tiff** (.tiff)
|
||||
* **image/webp** (.webp)
|
||||
* **video/x-msvideo** (.avi)
|
||||
|
|
@ -90,6 +91,7 @@ Now:
|
|||
* **application/pdf** (.pdf)
|
||||
* **application/x-photoshop** (.psd)
|
||||
* **application/clip** (.clip)
|
||||
* **application/x-krita** (.kra, .krz)
|
||||
* **application/sai2** (.sai2)
|
||||
* **application/vnd.rar** (.rar)
|
||||
* **application/zip** (.zip)
|
||||
|
|
|
|||
|
|
@ -34,6 +34,44 @@
|
|||
<div class="content">
|
||||
<h1 id="changelog"><a href="#changelog">changelog</a></h1>
|
||||
<ul>
|
||||
<li>
|
||||
<h2 id="version_535"><a href="#version_535">version 535</a></h2>
|
||||
<ul>
|
||||
<li><h3>misc</h3></li>
|
||||
<li>thanks to a user, we now have Krita (.kra, .krz) support! it even pulls thumbnails]</li>
|
||||
<li>thanks to another user, we now have SVG (.svg) support! it even generates thumbnails!</li>
|
||||
<li>I think I fixed a comparison statement calculator divide-by-zero error in the duplicate filter when you compare a file with a resolution with a file without one</li>
|
||||
<li><h3>petitions overview</h3></li>
|
||||
<li>_this is a workflow/usability update only for server janitors_</li>
|
||||
<li>tl;dr: the petitions page now fetches many petitions at once. update your servers and clients for it all to work right</li>
|
||||
<li>so, the petitions page now fetches lots of petitions with each 'fetch' button click. you can set how many it will fetch with a new number control</li>
|
||||
<li>the petitions are shown in a new multi-column list that shows action, account id, reason, and total weight. the actual data for the petitions will load in quickly, reflected in the list. as soon as the first is loaded, it is highlighted, but double-click any to highlight it in the old petition UI as normal</li>
|
||||
<li>when you process petitions, the client moves instantly to the next, all fitting into the existing workflow, without having to wait for the server to fetch a new one after you commit</li>
|
||||
<li>you can also mass approve/deny from here! if one account is doing great or terrible stuff, you can now blang it all in one go</li>
|
||||
<li><h3>petitions details</h3></li>
|
||||
<li>the 'fetch x petition' buttons now show `(*)` in their label if they are the active petition type being worked on</li>
|
||||
<li>petition pages now remember: the last petition type they were looking at; the number of petitions to fetch; and the number of files to show</li>
|
||||
<li>the petition page will pause any ongoing petition fetches if you close it, and resume if you unclose it</li>
|
||||
<li>a system where multi-mapping petitions would be broken up and delivered in tags with weight-similar chunks (e.g. if would say 'aaa for 11 files' and 'bbb in 15 files' in the same fetch, but not 'ccc in 542,154 files') is abandoned. this was not well explained and was causing confusion and code complexity. these petitions now appear clientside in full</li>
|
||||
<li>another system, where multi-mapping petitions would be delivered in same-namespace chunks, is also abandoned, for similar reasons. it was causing more confusion, especially when compared to the newer petition counting tech I've added. perhaps it will come back in as a clientside filter option</li>
|
||||
<li>the list of petitions you are given _should_ also be neatly grouped by account id, so rather than randomly sampling from all petitions, you'll get batches by user x, y, or z, and in most cases you'll be looking at everything by user x, and y, and then z up to the limit of num petitions you chose to fetch</li>
|
||||
<li>drawback: since petitions' content can overlap in complicated ways, and janitors can work on the same list at the same time, in edge cases the list you see can be slightly out of sync with what the server actually has. this isn't a big deal, and the worst case is wasted work as you approve the same thing twice. I tried to implement 'refresh list if count drops more than expected' tech, but the situation is complicated and it was spamming too much. I will let you refresh the list with a button click yourself for now, as you like, and please let me know where it works and fails</li>
|
||||
<li>drawback: I added some new objects, so you have to update both server and client for this to work. older/newer combinations will give you some harmless errors</li>
|
||||
<li>also, if your list starts running low, but there are plenty more petitions to work on, it will auto-refresh. again, it won't interrupt your current work, but it will fetch more. let me know how it works out</li>
|
||||
<li>drawback: while the new petition summary list is intentionally lightweight, I do spend some extra CPU figuring it out. with a high 'num petitions to fetch', it may take several seconds for a very busy server like the PTR just to fetch the initial list, so please play around with different fetch sizes and let me know what works well and what is way too slow</li>
|
||||
<li>there are still some things I want to do to this page, which I want to slip in the near future. I want to hide/show the sort and 'num files to show' widgets as appropriate, figure out a right-click menu for the new list to retry failures, and get some shortcut support going</li>
|
||||
<li><h3>boring code cleanup</h3></li>
|
||||
<li>wrote a new petition header object to hold content type, petition status, account id, and reason for petitions</li>
|
||||
<li>serverside petition fetching is now split into 'get petition headers' and 'get petition data'. the 'headers' section supports filtering by account id and in future reason</li>
|
||||
<li>the clientside petition management UI code pretty much got a full pass</li>
|
||||
<li>cleaned a bunch of ancient server db code</li>
|
||||
<li>cleaned a bunch of the clientside petition code. it was a real tangle</li>
|
||||
<li>improved the resilience of the hydrus server when it is given unacceptable tags in a content update</li>
|
||||
<li>all fetches of multiple rows of data from multi-column lists now happen sorted. this is just a little thing, but it'll probably dejank a few operations where you edit several things at once or get some errors and are trying to figure out which of five things caused it</li>
|
||||
<li>the hydrus official mimetype for psd files is now 'image/vnd.adobe.photoshop' (instead of 'application/x-photoshop')</li>
|
||||
<li>with krita file (which are actually just zip files) support, we now have the very barebones of archive tech started. I'll expand it a bit more and we should be able to improve support for other archive-like formats in the future</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<h2 id="version_534"><a href="#version_534">version 534</a></h2>
|
||||
<ul>
|
||||
|
|
|
|||
|
|
@ -1444,6 +1444,14 @@ class Controller( HydrusController.HydrusController ):
|
|||
|
||||
|
||||
|
||||
def PageAliveAndNotClosed( self, page_key ):
|
||||
|
||||
with self._page_key_lock:
|
||||
|
||||
return page_key in self._alive_page_keys and page_key not in self._closed_page_keys
|
||||
|
||||
|
||||
|
||||
def PageClosedButNotDestroyed( self, page_key ):
|
||||
|
||||
with self._page_key_lock:
|
||||
|
|
@ -1452,6 +1460,14 @@ class Controller( HydrusController.HydrusController ):
|
|||
|
||||
|
||||
|
||||
def PageDestroyed( self, page_key ):
|
||||
|
||||
with self._page_key_lock:
|
||||
|
||||
return page_key not in self._alive_page_keys and page_key not in self._closed_page_keys
|
||||
|
||||
|
||||
|
||||
def PrepStringForDisplay( self, text ):
|
||||
|
||||
return text.lower()
|
||||
|
|
|
|||
|
|
@ -120,61 +120,66 @@ def GetDuplicateComparisonStatements( shown_media, comparison_media ):
|
|||
|
||||
if s_size != c_size:
|
||||
|
||||
absolute_size_ratio = max( s_size, c_size ) / min( s_size, c_size )
|
||||
all_measurements_are_good = None not in ( s_size, c_size ) and True not in ( d <= 0 for d in ( s_size, c_size ) )
|
||||
|
||||
if absolute_size_ratio > 2.0:
|
||||
if all_measurements_are_good:
|
||||
|
||||
if s_size > c_size:
|
||||
absolute_size_ratio = max( s_size, c_size ) / min( s_size, c_size )
|
||||
|
||||
if absolute_size_ratio > 2.0:
|
||||
|
||||
operator = '>>'
|
||||
score = duplicate_comparison_score_much_higher_filesize
|
||||
if s_size > c_size:
|
||||
|
||||
operator = '>>'
|
||||
score = duplicate_comparison_score_much_higher_filesize
|
||||
|
||||
else:
|
||||
|
||||
operator = '<<'
|
||||
score = -duplicate_comparison_score_much_higher_filesize
|
||||
|
||||
|
||||
elif absolute_size_ratio > 1.05:
|
||||
|
||||
if s_size > c_size:
|
||||
|
||||
operator = '>'
|
||||
score = duplicate_comparison_score_higher_filesize
|
||||
|
||||
else:
|
||||
|
||||
operator = '<'
|
||||
score = -duplicate_comparison_score_higher_filesize
|
||||
|
||||
|
||||
else:
|
||||
|
||||
operator = '<<'
|
||||
score = -duplicate_comparison_score_much_higher_filesize
|
||||
operator = CC.UNICODE_ALMOST_EQUAL_TO
|
||||
score = 0
|
||||
|
||||
|
||||
elif absolute_size_ratio > 1.05:
|
||||
|
||||
if s_size > c_size:
|
||||
|
||||
operator = '>'
|
||||
score = duplicate_comparison_score_higher_filesize
|
||||
sign = '+'
|
||||
percentage_difference = ( s_size / c_size ) - 1.0
|
||||
|
||||
else:
|
||||
|
||||
operator = '<'
|
||||
score = -duplicate_comparison_score_higher_filesize
|
||||
sign = ''
|
||||
percentage_difference = ( s_size / c_size ) - 1.0
|
||||
|
||||
|
||||
else:
|
||||
percentage_different_string = ' ({}{})'.format( sign, HydrusData.ConvertFloatToPercentage( percentage_difference ) )
|
||||
|
||||
operator = CC.UNICODE_ALMOST_EQUAL_TO
|
||||
score = 0
|
||||
if is_a_pixel_dupe:
|
||||
|
||||
score = 0
|
||||
|
||||
|
||||
|
||||
if s_size > c_size:
|
||||
statement = '{} {} {}{}'.format( HydrusData.ToHumanBytes( s_size ), operator, HydrusData.ToHumanBytes( c_size ), percentage_different_string )
|
||||
|
||||
sign = '+'
|
||||
percentage_difference = ( s_size / c_size ) - 1.0
|
||||
statements_and_scores[ 'filesize' ] = ( statement, score )
|
||||
|
||||
else:
|
||||
|
||||
sign = ''
|
||||
percentage_difference = ( s_size / c_size ) - 1.0
|
||||
|
||||
|
||||
percentage_different_string = ' ({}{})'.format( sign, HydrusData.ConvertFloatToPercentage( percentage_difference ) )
|
||||
|
||||
if is_a_pixel_dupe:
|
||||
|
||||
score = 0
|
||||
|
||||
|
||||
statement = '{} {} {}{}'.format( HydrusData.ToHumanBytes( s_size ), operator, HydrusData.ToHumanBytes( c_size ), percentage_different_string )
|
||||
|
||||
statements_and_scores[ 'filesize' ] = ( statement, score )
|
||||
|
||||
|
||||
# higher/same res
|
||||
|
|
@ -433,7 +438,7 @@ def GetDuplicateComparisonStatements( shown_media, comparison_media ):
|
|||
|
||||
if s_label != c_label:
|
||||
|
||||
if c_jpeg_quality is None or s_jpeg_quality is None:
|
||||
if c_jpeg_quality is None or s_jpeg_quality is None or c_jpeg_quality <= 0 or s_jpeg_quality <= 0:
|
||||
|
||||
score = 0
|
||||
|
||||
|
|
|
|||
|
|
@ -7,7 +7,6 @@ from qtpy import QtWidgets as QW
|
|||
from hydrus.core import HydrusData
|
||||
from hydrus.core import HydrusExceptions
|
||||
from hydrus.core import HydrusGlobals as HG
|
||||
from hydrus.core import HydrusTime
|
||||
|
||||
from hydrus.client.gui import QtPorting as QP
|
||||
from hydrus.client.gui.widgets import ClientGUICommon
|
||||
|
|
|
|||
|
|
@ -1528,3 +1528,22 @@ register_column_type( COLUMN_LIST_DOMAIN_MODIFIED_TIMESTAMPS.ID, COLUMN_LIST_DOM
|
|||
register_column_type( COLUMN_LIST_DOMAIN_MODIFIED_TIMESTAMPS.ID, COLUMN_LIST_DOMAIN_MODIFIED_TIMESTAMPS.TIMESTAMP, 'time', False, 23, True )
|
||||
|
||||
default_column_list_sort_lookup[ COLUMN_LIST_DOMAIN_MODIFIED_TIMESTAMPS.ID ] = ( COLUMN_LIST_DOMAIN_MODIFIED_TIMESTAMPS.DOMAIN, True )
|
||||
|
||||
class COLUMN_LIST_PETITIONS_SUMMARY( COLUMN_LIST_DEFINITION ):
|
||||
|
||||
ID = 71
|
||||
|
||||
ACTION = 0
|
||||
ACCOUNT_KEY = 1
|
||||
REASON = 2
|
||||
CONTENT = 3
|
||||
|
||||
|
||||
column_list_type_name_lookup[ COLUMN_LIST_PETITIONS_SUMMARY.ID ] = 'petitions summary'
|
||||
|
||||
register_column_type( COLUMN_LIST_PETITIONS_SUMMARY.ID, COLUMN_LIST_PETITIONS_SUMMARY.ACTION, 'action', False, 9, True )
|
||||
register_column_type( COLUMN_LIST_PETITIONS_SUMMARY.ID, COLUMN_LIST_PETITIONS_SUMMARY.ACCOUNT_KEY, 'account', False, 10, True )
|
||||
register_column_type( COLUMN_LIST_PETITIONS_SUMMARY.ID, COLUMN_LIST_PETITIONS_SUMMARY.REASON, 'reason', False, 15, True )
|
||||
register_column_type( COLUMN_LIST_PETITIONS_SUMMARY.ID, COLUMN_LIST_PETITIONS_SUMMARY.CONTENT, 'content', False, 16, True )
|
||||
|
||||
default_column_list_sort_lookup[ COLUMN_LIST_PETITIONS_SUMMARY.ID ] = ( COLUMN_LIST_PETITIONS_SUMMARY.ACCOUNT_KEY, True )
|
||||
|
|
|
|||
|
|
@ -329,7 +329,7 @@ class BetterListCtrl( QW.QTreeWidget ):
|
|||
return ( display_tuple, sort_tuple )
|
||||
|
||||
|
||||
def _GetSelected( self ):
|
||||
def _GetSelectedIndices( self ) -> typing.List[ int ]:
|
||||
|
||||
indices = []
|
||||
|
||||
|
|
@ -520,7 +520,12 @@ class BetterListCtrl( QW.QTreeWidget ):
|
|||
|
||||
def DeleteDatas( self, datas: typing.Iterable[ object ] ):
|
||||
|
||||
deletees = [ ( self._data_to_indices[ data ], data ) for data in datas ]
|
||||
deletees = [ ( self._data_to_indices[ data ], data ) for data in datas if data in self._data_to_indices ]
|
||||
|
||||
if len( deletees ) == 0:
|
||||
|
||||
return
|
||||
|
||||
|
||||
deletees.sort( reverse = True )
|
||||
|
||||
|
|
@ -552,7 +557,7 @@ class BetterListCtrl( QW.QTreeWidget ):
|
|||
|
||||
def DeleteSelected( self ):
|
||||
|
||||
indices = self._GetSelected()
|
||||
indices = self._GetSelectedIndices()
|
||||
|
||||
indices.sort( reverse = True )
|
||||
|
||||
|
|
@ -655,13 +660,15 @@ class BetterListCtrl( QW.QTreeWidget ):
|
|||
|
||||
if only_selected:
|
||||
|
||||
indices = self._GetSelected()
|
||||
indices = self._GetSelectedIndices()
|
||||
|
||||
else:
|
||||
|
||||
indices = list( self._indices_to_data_info.keys() )
|
||||
|
||||
|
||||
indices.sort()
|
||||
|
||||
result = []
|
||||
|
||||
for index in indices:
|
||||
|
|
|
|||
|
|
@ -166,6 +166,11 @@ def CreateManagementControllerPetitions( petition_service_key ):
|
|||
|
||||
management_controller.SetVariable( 'petition_service_key', petition_service_key )
|
||||
|
||||
management_controller.SetVariable( 'petition_type_content_type', None )
|
||||
management_controller.SetVariable( 'petition_type_status', None )
|
||||
management_controller.SetVariable( 'num_petitions_to_fetch', 40 )
|
||||
management_controller.SetVariable( 'num_files_to_show', 256 )
|
||||
|
||||
return management_controller
|
||||
|
||||
|
||||
|
|
@ -188,7 +193,7 @@ class ManagementController( HydrusSerialisable.SerialisableBase ):
|
|||
|
||||
SERIALISABLE_TYPE = HydrusSerialisable.SERIALISABLE_TYPE_MANAGEMENT_CONTROLLER
|
||||
SERIALISABLE_NAME = 'Client Page Management Controller'
|
||||
SERIALISABLE_VERSION = 12
|
||||
SERIALISABLE_VERSION = 13
|
||||
|
||||
def __init__( self, page_name = 'page' ):
|
||||
|
||||
|
|
@ -536,6 +541,27 @@ class ManagementController( HydrusSerialisable.SerialisableBase ):
|
|||
return ( 12, new_serialisable_info )
|
||||
|
||||
|
||||
if version == 12:
|
||||
|
||||
( page_name, management_type, serialisable_variables ) = old_serialisable_info
|
||||
|
||||
if management_type == MANAGEMENT_TYPE_PETITIONS:
|
||||
|
||||
variables = HydrusSerialisable.CreateFromSerialisableTuple( serialisable_variables )
|
||||
|
||||
variables[ 'petition_type_content_type' ] = None
|
||||
variables[ 'petition_type_status' ] = None
|
||||
variables[ 'num_petitions_to_fetch' ] = 40
|
||||
variables[ 'num_files_to_show' ] = 256
|
||||
|
||||
serialisable_variables = variables.GetSerialisableTuple()
|
||||
|
||||
|
||||
new_serialisable_info = ( page_name, management_type, serialisable_variables )
|
||||
|
||||
return ( 13, new_serialisable_info )
|
||||
|
||||
|
||||
|
||||
def GetAPIInfoDict( self, simple ):
|
||||
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load Diff
|
|
@ -1446,7 +1446,7 @@ class ListBook( QW.QWidget ):
|
|||
|
||||
|
||||
class NoneableSpinCtrl( QW.QWidget ):
|
||||
|
||||
|
||||
valueChanged = QC.Signal()
|
||||
|
||||
def __init__( self, parent, message = '', none_phrase = 'no limit', min = 0, max = 1000000, unit = None, multiplier = 1, num_dimensions = 1 ):
|
||||
|
|
|
|||
|
|
@ -0,0 +1,26 @@
|
|||
import zipfile
|
||||
|
||||
def ExtractSingleFileFromZip( path_to_zip, filename_to_extract, extract_into_file_path ):
|
||||
|
||||
with zipfile.ZipFile( path_to_zip ) as zip_handle:
|
||||
|
||||
with zip_handle.open( filename_to_extract ) as reader:
|
||||
|
||||
with open( extract_into_file_path, "wb" ) as writer:
|
||||
|
||||
writer.write( reader.read() )
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
def ReadSingleFileFromZip( path_to_zip, filename_to_extract ):
|
||||
|
||||
with zipfile.ZipFile( path_to_zip ) as zip_handle:
|
||||
|
||||
with zip_handle.open( filename_to_extract ) as reader:
|
||||
|
||||
return reader.read()
|
||||
|
||||
|
||||
|
||||
|
|
@ -100,7 +100,7 @@ options = {}
|
|||
# Misc
|
||||
|
||||
NETWORK_VERSION = 20
|
||||
SOFTWARE_VERSION = 534
|
||||
SOFTWARE_VERSION = 535
|
||||
CLIENT_API_VERSION = 48
|
||||
|
||||
SERVER_THUMBNAIL_DIMENSIONS = ( 200, 200 )
|
||||
|
|
@ -794,6 +794,7 @@ mime_enum_lookup = {
|
|||
'application/x-shockwave-flash' : APPLICATION_FLASH,
|
||||
'application/x-photoshop' : APPLICATION_PSD,
|
||||
'image/vnd.adobe.photoshop' : APPLICATION_PSD,
|
||||
'application/vnd.adobe.photoshop' : APPLICATION_PSD,
|
||||
'application/clip' : APPLICATION_CLIP,
|
||||
'application/sai2': APPLICATION_SAI2,
|
||||
'application/x-krita': APPLICATION_KRITA,
|
||||
|
|
@ -855,10 +856,10 @@ mime_string_lookup = {
|
|||
APPLICATION_JSON : 'json',
|
||||
APPLICATION_CBOR : 'cbor',
|
||||
APPLICATION_PDF : 'pdf',
|
||||
APPLICATION_PSD : 'photoshop psd',
|
||||
APPLICATION_PSD : 'psd',
|
||||
APPLICATION_CLIP : 'clip',
|
||||
APPLICATION_SAI2 : 'sai2',
|
||||
APPLICATION_KRITA : 'kra',
|
||||
APPLICATION_KRITA : 'krita',
|
||||
APPLICATION_ZIP : 'zip',
|
||||
APPLICATION_RAR : 'rar',
|
||||
APPLICATION_7Z : '7z',
|
||||
|
|
@ -917,7 +918,7 @@ mime_mimetype_string_lookup = {
|
|||
APPLICATION_JSON : 'application/json',
|
||||
APPLICATION_CBOR : 'application/cbor',
|
||||
APPLICATION_PDF : 'application/pdf',
|
||||
APPLICATION_PSD : 'application/x-photoshop',
|
||||
APPLICATION_PSD : 'image/vnd.adobe.photoshop',
|
||||
APPLICATION_CLIP : 'application/clip',
|
||||
APPLICATION_SAI2: 'application/sai2',
|
||||
APPLICATION_KRITA: 'application/x-krita',
|
||||
|
|
|
|||
|
|
@ -485,6 +485,18 @@ def GetMime( path, ok_to_look_for_hydrus_updates = False ):
|
|||
|
||||
if it_passes:
|
||||
|
||||
if mime == HC.APPLICATION_ZIP:
|
||||
|
||||
# TODO: since we'll be expanding this to other zip-likes, we should make the zipfile object up here and pass that to various checkers downstream
|
||||
if HydrusKritaHandling.ZipLooksLikeAKrita( path ):
|
||||
|
||||
return HC.APPLICATION_KRITA
|
||||
|
||||
else:
|
||||
|
||||
return HC.APPLICATION_ZIP
|
||||
|
||||
|
||||
if mime in ( HC.UNDETERMINED_WM, HC.UNDETERMINED_MP4 ):
|
||||
|
||||
return HydrusVideoHandling.GetMime( path )
|
||||
|
|
@ -511,9 +523,11 @@ def GetMime( path, ok_to_look_for_hydrus_updates = False ):
|
|||
|
||||
return HC.TEXT_HTML
|
||||
|
||||
|
||||
if HydrusText.LooksLikeSVG( bit_to_check ):
|
||||
|
||||
return HC.IMAGE_SVG
|
||||
|
||||
|
||||
# it is important this goes at the end, because ffmpeg has a billion false positives!
|
||||
# for instance, it once thought some hydrus update files were mpegs
|
||||
|
|
|
|||
|
|
@ -1,93 +1,99 @@
|
|||
|
||||
from hydrus.core import HydrusArchiveHandling
|
||||
from hydrus.core import HydrusExceptions
|
||||
from hydrus.core import HydrusTemp
|
||||
|
||||
import zipfile
|
||||
import re
|
||||
|
||||
KRITA_FILE_THUMB = "preview.png"
|
||||
KRITA_FILE_MERGED = "mergedimage.png"
|
||||
|
||||
|
||||
def ExtractSingleFileFromZip( path_to_zip, filename_to_extract, extract_into_file_path ):
|
||||
|
||||
with zipfile.ZipFile( path_to_zip ) as zip_handle:
|
||||
|
||||
with zip_handle.open( filename_to_extract ) as reader:
|
||||
|
||||
with open( extract_into_file_path, "wb" ) as writer:
|
||||
|
||||
writer.write( reader.read() )
|
||||
|
||||
KRITA_MIMETYPE = 'mimetype'
|
||||
|
||||
def ExtractZippedImageToPath( path_to_zip, temp_path_file ):
|
||||
|
||||
|
||||
try:
|
||||
|
||||
ExtractSingleFileFromZip( path_to_zip, KRITA_FILE_MERGED, temp_path_file )
|
||||
|
||||
HydrusArchiveHandling.ExtractSingleFileFromZip( path_to_zip, KRITA_FILE_MERGED, temp_path_file )
|
||||
|
||||
return
|
||||
|
||||
except KeyError:
|
||||
|
||||
pass
|
||||
|
||||
|
||||
pass
|
||||
|
||||
|
||||
try:
|
||||
|
||||
ExtractSingleFileFromZip( path_to_zip, KRITA_FILE_THUMB, temp_path_file )
|
||||
HydrusArchiveHandling.ExtractSingleFileFromZip( path_to_zip, KRITA_FILE_THUMB, temp_path_file )
|
||||
|
||||
except KeyError:
|
||||
|
||||
|
||||
raise HydrusExceptions.DamagedOrUnusualFileException( f'This krita file had no {KRITA_FILE_MERGED} or {KRITA_FILE_THUMB}, so no PNG thumb could be extracted!' )
|
||||
|
||||
|
||||
|
||||
|
||||
# TODO: animation and frame stuff which is also in the maindoc.xml
|
||||
def GetKraProperties( path ):
|
||||
|
||||
|
||||
( os_file_handle, maindoc_xml ) = HydrusTemp.GetTempPath()
|
||||
|
||||
|
||||
DOCUMENT_INFO_FILE = "maindoc.xml"
|
||||
|
||||
|
||||
# TODO: probably actually parse the xml instead of using regex
|
||||
FIND_KEY_VALUE = re.compile(r"([a-z\-\_]+)\s*=\s*['\"]([^'\"]+)", re.IGNORECASE)
|
||||
|
||||
FIND_KEY_VALUE = re.compile(r"([a-z\-_]+)\s*=\s*['\"]([^'\"]+)", re.IGNORECASE)
|
||||
|
||||
width = None
|
||||
height = None
|
||||
|
||||
|
||||
try:
|
||||
|
||||
ExtractSingleFileFromZip( path, DOCUMENT_INFO_FILE, maindoc_xml )
|
||||
|
||||
|
||||
HydrusArchiveHandling.ExtractSingleFileFromZip( path, DOCUMENT_INFO_FILE, maindoc_xml )
|
||||
|
||||
with open(maindoc_xml, "r") as reader:
|
||||
|
||||
|
||||
for line in reader:
|
||||
|
||||
|
||||
for match in FIND_KEY_VALUE.findall( line ):
|
||||
|
||||
|
||||
key, value = match
|
||||
|
||||
|
||||
if key == "width" and value.isdigit():
|
||||
|
||||
|
||||
width = int(value)
|
||||
|
||||
|
||||
if key == "height" and value.isdigit():
|
||||
|
||||
|
||||
height = int(value)
|
||||
|
||||
|
||||
if width is not None and height is not None:
|
||||
|
||||
|
||||
break
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
except KeyError:
|
||||
|
||||
|
||||
raise HydrusExceptions.DamagedOrUnusualFileException( f'This krita file had no {DOCUMENT_INFO_FILE}, so no information could be extracted!' )
|
||||
|
||||
|
||||
finally:
|
||||
|
||||
|
||||
HydrusTemp.CleanUpTempPath( os_file_handle, maindoc_xml )
|
||||
|
||||
|
||||
|
||||
return width, height
|
||||
|
||||
|
||||
|
||||
|
||||
def ZipLooksLikeAKrita( path ):
|
||||
|
||||
try:
|
||||
|
||||
mimetype_data = HydrusArchiveHandling.ReadSingleFileFromZip( path, KRITA_MIMETYPE )
|
||||
|
||||
return b'application/x-krita' in mimetype_data
|
||||
|
||||
except KeyError:
|
||||
|
||||
return False
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -139,6 +139,7 @@ SERIALISABLE_TYPE_METADATA_SINGLE_FILE_IMPORTER_MEDIA_NOTES = 120
|
|||
SERIALISABLE_TYPE_TIMESTAMP_DATA = 121
|
||||
SERIALISABLE_TYPE_METADATA_SINGLE_FILE_EXPORTER_MEDIA_TIMESTAMPS = 122
|
||||
SERIALISABLE_TYPE_METADATA_SINGLE_FILE_IMPORTER_MEDIA_TIMESTAMPS = 123
|
||||
SERIALISABLE_TYPE_PETITION_HEADER = 124
|
||||
|
||||
SERIALISABLE_TYPES_TO_OBJECT_TYPES = {}
|
||||
|
||||
|
|
|
|||
|
|
@ -114,8 +114,7 @@ def LooksLikeHTML( file_data ):
|
|||
return False
|
||||
|
||||
def LooksLikeSVG( file_data ):
|
||||
|
||||
|
||||
|
||||
if isinstance( file_data, bytes ):
|
||||
|
||||
search_elements = ( b'<svg', b'<SVG', b'<!DOCTYPE svg', b'<!DOCTYPE SVG' )
|
||||
|
|
@ -135,6 +134,7 @@ def LooksLikeSVG( file_data ):
|
|||
|
||||
return False
|
||||
|
||||
|
||||
def LooksLikeJSON( file_data ):
|
||||
|
||||
try:
|
||||
|
|
|
|||
|
|
@ -1420,6 +1420,18 @@ class Content( HydrusSerialisable.SerialisableBase ):
|
|||
return hashes
|
||||
|
||||
|
||||
def GetActualWeight( self ):
|
||||
|
||||
if self._content_type in ( HC.CONTENT_TYPE_FILES, HC.CONTENT_TYPE_MAPPINGS ):
|
||||
|
||||
return len( self.GetHashes() )
|
||||
|
||||
else:
|
||||
|
||||
return 1
|
||||
|
||||
|
||||
|
||||
def GetVirtualWeight( self ):
|
||||
|
||||
if self._content_type in ( HC.CONTENT_TYPE_FILES, HC.CONTENT_TYPE_MAPPINGS ):
|
||||
|
|
@ -1462,7 +1474,7 @@ class Content( HydrusSerialisable.SerialisableBase ):
|
|||
|
||||
( tag, hashes ) = self._content_data
|
||||
|
||||
for chunk_of_hashes in HydrusLists.SplitListIntoChunks( hashes, 100 ):
|
||||
for chunk_of_hashes in HydrusLists.SplitListIntoChunks( hashes, 500 ):
|
||||
|
||||
content = Content( content_type = self._content_type, content_data = ( tag, chunk_of_hashes ) )
|
||||
|
||||
|
|
@ -2311,15 +2323,16 @@ class Metadata( HydrusSerialisable.SerialisableBase ):
|
|||
|
||||
|
||||
|
||||
|
||||
HydrusSerialisable.SERIALISABLE_TYPES_TO_OBJECT_TYPES[ HydrusSerialisable.SERIALISABLE_TYPE_METADATA ] = Metadata
|
||||
|
||||
class Petition( HydrusSerialisable.SerialisableBase ):
|
||||
|
||||
SERIALISABLE_TYPE = HydrusSerialisable.SERIALISABLE_TYPE_PETITION
|
||||
SERIALISABLE_NAME = 'Petition'
|
||||
SERIALISABLE_VERSION = 2
|
||||
SERIALISABLE_VERSION = 3
|
||||
|
||||
def __init__( self, petitioner_account = None, reason = None, actions_and_contents = None ):
|
||||
def __init__( self, petitioner_account = None, petition_header = None, actions_and_contents = None ):
|
||||
|
||||
if actions_and_contents is None:
|
||||
|
||||
|
|
@ -2329,23 +2342,42 @@ class Petition( HydrusSerialisable.SerialisableBase ):
|
|||
HydrusSerialisable.SerialisableBase.__init__( self )
|
||||
|
||||
self._petitioner_account = petitioner_account
|
||||
self._reason = reason
|
||||
self._petition_header = petition_header
|
||||
self._actions_and_contents = [ ( action, HydrusSerialisable.SerialisableList( contents ) ) for ( action, contents ) in actions_and_contents ]
|
||||
|
||||
self._completed_actions_to_contents = collections.defaultdict( list )
|
||||
|
||||
|
||||
def __eq__( self, other ):
|
||||
|
||||
if isinstance( other, Petition ):
|
||||
|
||||
return self.__hash__() == other.__hash__()
|
||||
|
||||
|
||||
return NotImplemented
|
||||
|
||||
|
||||
def __hash__( self ):
|
||||
|
||||
return self._petition_header.__hash__()
|
||||
|
||||
|
||||
def _GetSerialisableInfo( self ):
|
||||
|
||||
serialisable_petitioner_account = Account.GenerateSerialisableTupleFromAccount( self._petitioner_account )
|
||||
serialisable_petition_header = self._petition_header.GetSerialisableTuple()
|
||||
serialisable_actions_and_contents = [ ( action, contents.GetSerialisableTuple() ) for ( action, contents ) in self._actions_and_contents ]
|
||||
|
||||
return ( serialisable_petitioner_account, self._reason, serialisable_actions_and_contents )
|
||||
return ( serialisable_petitioner_account, serialisable_petition_header, serialisable_actions_and_contents )
|
||||
|
||||
|
||||
def _InitialiseFromSerialisableInfo( self, serialisable_info ):
|
||||
|
||||
( serialisable_petitioner_account, self._reason, serialisable_actions_and_contents ) = serialisable_info
|
||||
( serialisable_petitioner_account, serialisable_petition_header, serialisable_actions_and_contents ) = serialisable_info
|
||||
|
||||
self._petitioner_account = Account.GenerateAccountFromSerialisableTuple( serialisable_petitioner_account )
|
||||
self._petition_header = HydrusSerialisable.CreateFromSerialisableTuple( serialisable_petition_header )
|
||||
self._actions_and_contents = [ ( action, HydrusSerialisable.CreateFromSerialisableTuple( serialisable_contents ) ) for ( action, serialisable_contents ) in serialisable_actions_and_contents ]
|
||||
|
||||
|
||||
|
|
@ -2366,6 +2398,151 @@ class Petition( HydrusSerialisable.SerialisableBase ):
|
|||
return ( 2, new_serialisable_info )
|
||||
|
||||
|
||||
if version == 3:
|
||||
|
||||
# we'll dump out, since this code should never be reached. a new client won't be receiving old petitions since an old server won't have the calls
|
||||
# it is appropriate to update the version though--that lets an old client talking to a new server get a nicer 'version from the future' error
|
||||
|
||||
raise NotImplementedError()
|
||||
|
||||
|
||||
|
||||
def Approve( self, action, content ):
|
||||
|
||||
self._completed_actions_to_contents[ action ].append( content )
|
||||
|
||||
|
||||
def ApproveAll( self ):
|
||||
|
||||
for ( action, contents ) in self._actions_and_contents:
|
||||
|
||||
for content in contents:
|
||||
|
||||
self.Approve( action, content )
|
||||
|
||||
|
||||
|
||||
|
||||
def Deny( self, action, content ):
|
||||
|
||||
if action == HC.CONTENT_UPDATE_PEND:
|
||||
|
||||
denial_action = HC.CONTENT_UPDATE_DENY_PEND
|
||||
|
||||
elif action == HC.CONTENT_UPDATE_PETITION:
|
||||
|
||||
denial_action = HC.CONTENT_UPDATE_DENY_PETITION
|
||||
|
||||
else:
|
||||
|
||||
raise Exception( 'Petition came with unexpected action: {}'.format( action ) )
|
||||
|
||||
|
||||
self._completed_actions_to_contents[ denial_action ].append( content )
|
||||
|
||||
|
||||
def DenyAll( self ):
|
||||
|
||||
for ( action, contents ) in self._actions_and_contents:
|
||||
|
||||
for content in contents:
|
||||
|
||||
self.Deny( action, content )
|
||||
|
||||
|
||||
|
||||
|
||||
def GetAllCompletedUpdates( self ):
|
||||
|
||||
def break_contents_into_chunks( some_contents ):
|
||||
|
||||
chunks_of_some_contents = []
|
||||
chunk_of_some_contents = []
|
||||
|
||||
weight_of_current_chunk = 0
|
||||
|
||||
for content in some_contents:
|
||||
|
||||
for content_chunk in content.IterateUploadableChunks(): # break 20K-strong mappings petitions into smaller bits to POST back
|
||||
|
||||
chunk_of_some_contents.append( content_chunk )
|
||||
|
||||
weight_of_current_chunk += content.GetVirtualWeight()
|
||||
|
||||
if weight_of_current_chunk > 50:
|
||||
|
||||
chunks_of_some_contents.append( chunk_of_some_contents )
|
||||
|
||||
chunk_of_some_contents = []
|
||||
|
||||
weight_of_current_chunk = 0
|
||||
|
||||
|
||||
|
||||
|
||||
if len( chunk_of_some_contents ) > 0:
|
||||
|
||||
chunks_of_some_contents.append( chunk_of_some_contents )
|
||||
|
||||
|
||||
return chunks_of_some_contents
|
||||
|
||||
|
||||
updates_and_content_updates = []
|
||||
|
||||
# make sure you delete before you add
|
||||
for action in ( HC.CONTENT_UPDATE_DENY_PETITION, HC.CONTENT_UPDATE_DENY_PEND, HC.CONTENT_UPDATE_PETITION, HC.CONTENT_UPDATE_PEND ):
|
||||
|
||||
contents = self._completed_actions_to_contents[ action ]
|
||||
|
||||
if len( contents ) == 0:
|
||||
|
||||
continue
|
||||
|
||||
|
||||
if action == HC.CONTENT_UPDATE_PEND:
|
||||
|
||||
content_update_action = HC.CONTENT_UPDATE_ADD
|
||||
|
||||
elif action == HC.CONTENT_UPDATE_PETITION:
|
||||
|
||||
content_update_action = HC.CONTENT_UPDATE_DELETE
|
||||
|
||||
else:
|
||||
|
||||
content_update_action = None
|
||||
|
||||
|
||||
chunks_of_contents = break_contents_into_chunks( contents )
|
||||
|
||||
for chunk_of_contents in chunks_of_contents:
|
||||
|
||||
update = ClientToServerUpdate()
|
||||
content_updates = []
|
||||
|
||||
for content in chunk_of_contents:
|
||||
|
||||
update.AddContent( action, content, self._petition_header.reason )
|
||||
|
||||
if content_update_action is not None:
|
||||
|
||||
content_type = content.GetContentType()
|
||||
|
||||
row = content.GetContentData()
|
||||
|
||||
content_update = HydrusData.ContentUpdate( content_type, content_update_action, row )
|
||||
|
||||
content_updates.append( content_update )
|
||||
|
||||
|
||||
|
||||
updates_and_content_updates.append( ( update, content_updates ) )
|
||||
|
||||
|
||||
|
||||
return updates_and_content_updates
|
||||
|
||||
|
||||
def GetContents( self, action ):
|
||||
|
||||
actions_to_contents = dict( self._actions_and_contents )
|
||||
|
|
@ -2390,74 +2567,116 @@ class Petition( HydrusSerialisable.SerialisableBase ):
|
|||
return self._petitioner_account
|
||||
|
||||
|
||||
def GetPetitionHeader( self ) -> "PetitionHeader":
|
||||
|
||||
return self._petition_header
|
||||
|
||||
|
||||
def GetReason( self ):
|
||||
|
||||
return self._reason
|
||||
return self._petition_header.reason
|
||||
|
||||
|
||||
@staticmethod
|
||||
def GetApproval( action, contents, reason ):
|
||||
def GetActualContentWeight( self ) -> int:
|
||||
|
||||
update = ClientToServerUpdate()
|
||||
content_updates = []
|
||||
total_weight = 0
|
||||
|
||||
if action == HC.CONTENT_UPDATE_PEND:
|
||||
for ( action, contents ) in self._actions_and_contents:
|
||||
|
||||
content_update_action = HC.CONTENT_UPDATE_ADD
|
||||
for content in contents:
|
||||
|
||||
total_weight += content.GetActualWeight()
|
||||
|
||||
|
||||
elif action == HC.CONTENT_UPDATE_PETITION:
|
||||
|
||||
return total_weight
|
||||
|
||||
|
||||
def GetContentSummary( self ) -> str:
|
||||
|
||||
num_sub_petitions = sum( ( len( contents ) for ( action, contents ) in self._actions_and_contents ) )
|
||||
|
||||
if self._petition_header.content_type == HC.CONTENT_TYPE_MAPPINGS and num_sub_petitions > 1:
|
||||
|
||||
content_update_action = HC.CONTENT_UPDATE_DELETE
|
||||
return '{} mappings in {} petitions'.format( HydrusData.ToHumanInt( self.GetActualContentWeight() ), HydrusData.ToHumanInt( num_sub_petitions ) )
|
||||
|
||||
else:
|
||||
|
||||
raise Exception( 'Petition came with unexpected action: {}'.format( action ) )
|
||||
return '{} {}'.format( HydrusData.ToHumanInt( self.GetActualContentWeight() ), HC.content_type_string_lookup[ self._petition_header.content_type ] )
|
||||
|
||||
|
||||
for content in contents:
|
||||
|
||||
update.AddContent( action, content, reason )
|
||||
|
||||
content_type = content.GetContentType()
|
||||
|
||||
row = content.GetContentData()
|
||||
|
||||
content_update = HydrusData.ContentUpdate( content_type, content_update_action, row )
|
||||
|
||||
content_updates.append( content_update )
|
||||
|
||||
|
||||
return ( update, content_updates )
|
||||
|
||||
|
||||
@staticmethod
|
||||
def GetDenial( action, contents, reason ):
|
||||
|
||||
update = ClientToServerUpdate()
|
||||
|
||||
if action == HC.CONTENT_UPDATE_PEND:
|
||||
|
||||
denial_action = HC.CONTENT_UPDATE_DENY_PEND
|
||||
|
||||
elif action == HC.CONTENT_UPDATE_PETITION:
|
||||
|
||||
denial_action = HC.CONTENT_UPDATE_DENY_PETITION
|
||||
|
||||
else:
|
||||
|
||||
raise Exception( 'Petition came with unexpected action: {}'.format( action ) )
|
||||
|
||||
|
||||
for content in contents:
|
||||
|
||||
update.AddContent( denial_action, content, reason )
|
||||
|
||||
|
||||
return update
|
||||
|
||||
|
||||
|
||||
HydrusSerialisable.SERIALISABLE_TYPES_TO_OBJECT_TYPES[ HydrusSerialisable.SERIALISABLE_TYPE_PETITION ] = Petition
|
||||
|
||||
class PetitionHeader( HydrusSerialisable.SerialisableBase ):
|
||||
|
||||
SERIALISABLE_TYPE = HydrusSerialisable.SERIALISABLE_TYPE_PETITION_HEADER
|
||||
SERIALISABLE_NAME = 'Petitions Header'
|
||||
SERIALISABLE_VERSION = 1
|
||||
|
||||
def __init__( self, content_type = None, status = None, account_key = None, reason = None ):
|
||||
|
||||
if content_type is None:
|
||||
|
||||
content_type = HC.CONTENT_TYPE_MAPPINGS
|
||||
|
||||
|
||||
if status is None:
|
||||
|
||||
status = HC.CONTENT_STATUS_PETITIONED
|
||||
|
||||
|
||||
if account_key is None:
|
||||
|
||||
account_key = b''
|
||||
|
||||
|
||||
if reason is None:
|
||||
|
||||
reason = ''
|
||||
|
||||
|
||||
HydrusSerialisable.SerialisableBase.__init__( self )
|
||||
|
||||
self.content_type = content_type
|
||||
self.status = status
|
||||
self.account_key = account_key
|
||||
self.reason = reason
|
||||
|
||||
|
||||
def __eq__( self, other ):
|
||||
|
||||
if isinstance( other, PetitionHeader ):
|
||||
|
||||
return self.__hash__() == other.__hash__()
|
||||
|
||||
|
||||
return NotImplemented
|
||||
|
||||
|
||||
def __hash__( self ):
|
||||
|
||||
return ( self.content_type, self.status, self.account_key, self.reason ).__hash__()
|
||||
|
||||
|
||||
def _GetSerialisableInfo( self ):
|
||||
|
||||
serialisable_account_key = self.account_key.hex()
|
||||
|
||||
return ( self.content_type, self.status, serialisable_account_key, self.reason )
|
||||
|
||||
|
||||
def _InitialiseFromSerialisableInfo( self, serialisable_info ):
|
||||
|
||||
( self.content_type, self.status, serialisable_account_key, self.reason ) = serialisable_info
|
||||
|
||||
self.account_key = bytes.fromhex( serialisable_account_key )
|
||||
|
||||
|
||||
|
||||
HydrusSerialisable.SERIALISABLE_TYPES_TO_OBJECT_TYPES[ HydrusSerialisable.SERIALISABLE_TYPE_PETITION_HEADER ] = PetitionHeader
|
||||
|
||||
class ServerService( object ):
|
||||
|
||||
def __init__( self, service_key, service_type, name, port, dictionary ):
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load Diff
|
|
@ -70,7 +70,7 @@ class HydrusServiceRepository( HydrusServiceRestricted ):
|
|||
|
||||
root.putChild( b'num_petitions', ServerServerResources.HydrusResourceRestrictedNumPetitions( self._service, HydrusServer.REMOTE_DOMAIN ) )
|
||||
root.putChild( b'petition', ServerServerResources.HydrusResourceRestrictedPetition( self._service, HydrusServer.REMOTE_DOMAIN ) )
|
||||
root.putChild( b'petition_summary_list', ServerServerResources.HydrusResourceRestrictedPetitionSummaryList( self._service, HydrusServer.REMOTE_DOMAIN ) )
|
||||
root.putChild( b'petitions_summary', ServerServerResources.HydrusResourceRestrictedPetitionsSummary( self._service, HydrusServer.REMOTE_DOMAIN ) )
|
||||
root.putChild( b'update', ServerServerResources.HydrusResourceRestrictedUpdate( self._service, HydrusServer.REMOTE_DOMAIN ) )
|
||||
#root.putChild( b'immediate_update', ServerServerResources.HydrusResourceRestrictedImmediateUpdate( self._service, HydrusServer.REMOTE_DOMAIN ) )
|
||||
root.putChild( b'metadata', ServerServerResources.HydrusResourceRestrictedMetadataUpdate( self._service, HydrusServer.REMOTE_DOMAIN ) )
|
||||
|
|
|
|||
|
|
@ -917,28 +917,7 @@ class HydrusResourceRestrictedNumPetitions( HydrusResourceRestricted ):
|
|||
return response_context
|
||||
|
||||
|
||||
class HydrusResourceRestrictedPetitionSummaryList( HydrusResourceRestricted ):
|
||||
|
||||
def _checkAccountPermissions( self, request: HydrusServerRequest.HydrusRequest ):
|
||||
|
||||
content_type = request.parsed_request_args[ 'content_type' ]
|
||||
|
||||
request.hydrus_account.CheckPermission( content_type, HC.PERMISSION_ACTION_MODERATE )
|
||||
|
||||
|
||||
def _threadDoGETJob( self, request: HydrusServerRequest.HydrusRequest ):
|
||||
|
||||
# fetch cached summary list
|
||||
# ( account_key, reason, size of petition )
|
||||
petition_summary_list = []
|
||||
|
||||
body = HydrusNetworkVariableHandling.DumpHydrusArgsToNetworkBytes( { 'petition_summary_list' : petition_summary_list } )
|
||||
|
||||
response_context = HydrusServerResources.ResponseContext( 200, body = body )
|
||||
|
||||
return response_context
|
||||
|
||||
|
||||
|
||||
class HydrusResourceRestrictedPetition( HydrusResourceRestricted ):
|
||||
|
||||
def _checkAccountPermissions( self, request: HydrusServerRequest.HydrusRequest ):
|
||||
|
|
@ -950,12 +929,38 @@ class HydrusResourceRestrictedPetition( HydrusResourceRestricted ):
|
|||
|
||||
def _threadDoGETJob( self, request: HydrusServerRequest.HydrusRequest ):
|
||||
|
||||
subject_account_key = request.parsed_request_args.GetValueOrNone( 'subject_account_key', bytes )
|
||||
# add reason to here some time, for when we eventually select petitions from a summary list of ( account, reason, size ) stuff
|
||||
content_type = request.parsed_request_args[ 'content_type' ]
|
||||
status = request.parsed_request_args[ 'status' ]
|
||||
subject_account_key = request.parsed_request_args.GetValueOrNone( 'subject_account_key', bytes )
|
||||
reason = request.parsed_request_args.GetValueOrNone( 'reason', str )
|
||||
|
||||
petition = HG.server_controller.Read( 'petition', self._service_key, request.hydrus_account, content_type, status, subject_account_key = subject_account_key )
|
||||
if subject_account_key is None or reason is None:
|
||||
|
||||
petitions_summary = HG.server_controller.Read( 'petitions_summary', self._service_key, request.hydrus_account, content_type, status, limit = 1, subject_account_key = subject_account_key )
|
||||
|
||||
if len( petitions_summary ) == 0:
|
||||
|
||||
if subject_account_key is None and reason is None:
|
||||
|
||||
raise HydrusExceptions.NotFoundException( f'Sorry, no petitions were found!' )
|
||||
|
||||
elif subject_account_key is None:
|
||||
|
||||
raise HydrusExceptions.NotFoundException( f'Sorry, no petitions were found for the given reason {reason}!' )
|
||||
|
||||
else:
|
||||
|
||||
raise HydrusExceptions.NotFoundException( 'Sorry, no petitions were found for the given account_key {}!'.format( subject_account_key.hex() ) )
|
||||
|
||||
|
||||
|
||||
petition_header = petitions_summary[0]
|
||||
|
||||
subject_account_key = petition_header.account_key
|
||||
reason = petition_header.reason
|
||||
|
||||
|
||||
petition = HG.server_controller.Read( 'petition', self._service_key, request.hydrus_account, content_type, status, subject_account_key, reason )
|
||||
|
||||
body = HydrusNetworkVariableHandling.DumpHydrusArgsToNetworkBytes( { 'petition' : petition } )
|
||||
|
||||
|
|
@ -964,6 +969,35 @@ class HydrusResourceRestrictedPetition( HydrusResourceRestricted ):
|
|||
return response_context
|
||||
|
||||
|
||||
|
||||
class HydrusResourceRestrictedPetitionsSummary( HydrusResourceRestricted ):
|
||||
|
||||
def _checkAccountPermissions( self, request: HydrusServerRequest.HydrusRequest ):
|
||||
|
||||
content_type = request.parsed_request_args[ 'content_type' ]
|
||||
|
||||
request.hydrus_account.CheckPermission( content_type, HC.PERMISSION_ACTION_MODERATE )
|
||||
|
||||
|
||||
def _threadDoGETJob( self, request: HydrusServerRequest.HydrusRequest ):
|
||||
|
||||
content_type = request.parsed_request_args.GetValue( 'content_type', int )
|
||||
status = request.parsed_request_args.GetValue( 'status', int )
|
||||
num = request.parsed_request_args.GetValue( 'num', int )
|
||||
|
||||
subject_account_key = request.parsed_request_args.GetValueOrNone( 'subject_account_key', bytes )
|
||||
reason = request.parsed_request_args.GetValueOrNone( 'reason', str )
|
||||
|
||||
petitions_summary = HG.server_controller.Read( 'petitions_summary', self._service_key, request.hydrus_account, content_type, status, num, subject_account_key = subject_account_key, reason = reason )
|
||||
|
||||
body = HydrusNetworkVariableHandling.DumpHydrusArgsToNetworkBytes( { 'petitions_summary' : petitions_summary } )
|
||||
|
||||
response_context = HydrusServerResources.ResponseContext( 200, body = body )
|
||||
|
||||
return response_context
|
||||
|
||||
|
||||
|
||||
class HydrusResourceRestrictedRegistrationKeys( HydrusResourceRestricted ):
|
||||
|
||||
def _checkAccountPermissions( self, request: HydrusServerRequest.HydrusRequest ):
|
||||
|
|
|
|||
|
|
@ -400,11 +400,18 @@ class TestServer( unittest.TestCase ):
|
|||
|
||||
# petition
|
||||
|
||||
petitioner_account = HydrusNetwork.Account.GenerateUnknownAccount()
|
||||
petitioner_account = HydrusNetwork.Account.GenerateUnknownAccount( HydrusData.GenerateKey() )
|
||||
reason = 'it sucks'
|
||||
actions_and_contents = ( HC.CONTENT_UPDATE_PETITION, [ HydrusNetwork.Content( HC.CONTENT_TYPE_FILES, [ HydrusData.GenerateKey() for i in range( 10 ) ] ) ] )
|
||||
|
||||
petition = HydrusNetwork.Petition( petitioner_account, reason, actions_and_contents )
|
||||
petition_header = HydrusNetwork.PetitionHeader(
|
||||
content_type = HC.CONTENT_TYPE_FILES,
|
||||
status = HC.CONTENT_STATUS_PETITIONED,
|
||||
account_key = petitioner_account.GetAccountKey(),
|
||||
reason = reason
|
||||
)
|
||||
|
||||
petition = HydrusNetwork.Petition( petitioner_account = petitioner_account, petition_header = petition_header, actions_and_contents = actions_and_contents )
|
||||
|
||||
HG.test_controller.SetRead( 'petition', petition )
|
||||
|
||||
|
|
@ -412,6 +419,12 @@ class TestServer( unittest.TestCase ):
|
|||
|
||||
self.assertEqual( response[ 'petition' ].GetSerialisableTuple(), petition.GetSerialisableTuple() )
|
||||
|
||||
HG.test_controller.SetRead( 'petition', petition )
|
||||
|
||||
response = service.Request( HC.GET, 'petition', { 'content_type' : HC.CONTENT_TYPE_FILES, 'status' : HC.CONTENT_UPDATE_PETITION, 'account_key' : petitioner_account.GetAccountKey(), reason : reason } )
|
||||
|
||||
self.assertEqual( response[ 'petition' ].GetSerialisableTuple(), petition.GetSerialisableTuple() )
|
||||
|
||||
# definitions
|
||||
|
||||
definitions_update = HydrusNetwork.DefinitionsUpdate()
|
||||
|
|
|
|||
Loading…
Reference in New Issue