RepoMirrors
/
hydrus
mirror of https://github.com/hydrusnetwork/hydrus.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Version 494

This commit is contained in:
Hydrus Network Developer 2022-08-03 15:59:51 -05:00
parent 05f614ba89
commit 890deb324e
43 changed files with 936 additions and 314 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
.github/workflows/release.yml vendored
View File

@ -249,6 +249,10 @@ jobs:
run: |
find dist/client/ -type f -name "*.pyc" -delete
while read line; do find dist/client/ -type f -name "${line}" -delete ; done < hydrus/static/build_files/linux/files_to_delete.txt
-
name: Remove Surplus File
run: |
rm -f dist/client/libxkbcommon.so*
-
name: Set Permissions
run: |

61
docs/changelog.md
View File

@ -3,6 +3,40 @@
!!! note
This is the new changelog, only the most recent builds. For all versions, see the [old changelog](old_changelog.html).
## [Version 494](https://github.com/hydrusnetwork/hydrus/releases/tag/v494)
### QT6
* thanks to a user's help, we are rolling out a Qt6 test build this week. we've been running Qt5 for a few years now. 6 is mostly a very large bugfix patch, and I am hopeful this update will relieve several legacy issues related to UI scale, colour support, draw flickering, and other unusual stuff. so far, it is working for me great. I'll be putting out joint 5 and 6 builds for 4-8 weeks, to iron out any big problems, and then I'll switch over to 6 releases exclusively. if you are an advanced user, please give it a go this or next week and let me know if you run into any traceback errors about deprecated method names or completely jank layout in the less used parts of the program
* the actual changes you'll see are mostly style, just slightly different font spacing, things like that. if you have a system-baked Qt5 style that hydrus magically inherits, this will no longer work, you need to get a Qt6 version of the style (although I understand this is happening already for the popular styles, so you may already have them)
* users on Windows 7 and similarly old OS versions are unable to run Qt6 programs, sorry!
* I intend to keep the code 5-compatible, and users who run from source can choose whichever version of Qt they prefer, as here in the help: https://hydrusnetwork.github.io/hydrus/running_from_source.html#qt
* the linux Qt6 build also goes up from ubuntu 18.04 to 20.04. let me know if you have any trouble, but it feels like it is time to update this too
### file import options overhaul
* I wanted to do note parsing this week, but when I reviewed the whole job, there wasn't enough time to do it properly. so, in prep for a cleaner introduction of 'note import options' next week, I am overhauling how the other import options do some stuff
* all file import options now support filetype filtering! it uses the same control as system:filetype or in import folders, but with some improved logic. on update, existing import folder filetype settings will be copied down to the file import options
* file import options now work on a similar 'default' system as tag import options. existing file import options will stay as-is, but new ones will begin in a 'use the default settings at time of import' state. those defaults are editable under _options->importing_. for now I am not adding a 'use this file import options default for this web domain' system, but it might happen in future. let's see how this all shakes out first
* the file import options button now has a right-click menu like the tag import options button
* the manage subscriptions panel now has a 'overwrite file import options' button to mass-set FIO
* cleaned up a bunch of old file import and import options code
### misc
* system:filetype now remembers meta filetypes better. if you select 'all video', it will now still select all video even if hydev adds support for a new video type in future. also if you select 'video + animations', it'll say that rather than listing out every possible specific-type
* fixed an issue where loading a favourite search wasn't always setting 'include current/pending' values on the buttons correct
* fixed up a status display in the gallery downloader and watcher pages--if you pause an importer while it is doing work, it now says 'pausing...' as its status until any current jobs are finished. it was giving empty text before, as if it were finished already
* fixed some unusual behaviour with downloader highlighting where the first query pended to an empty page was secretly highlighted for the next session load, and fixed the 'subscription gap downloader' also doing this and not obeying the normal 'highlight new downloaders if nothing already highlighted' option
* improved the error when the 'make sure this directory exists' function runs into a file with that pathname
* fixed a rare selection position error, maybe Qt6 only, when clicking in the thumbnail grid as it is loading
### boring Qt6 code cleanup
* as a side thing, I set up quick-launch environments for QtPy5, QtPy6, PySide2, and PySide6 in my IDE this week, so I can now test all these situations and jump back in time no problem in future
* integrated a user's patch to bring us up to Qt6 compatibility and did a little more work to get it backwards compatible with older qtpy and Qt5
* refactored the critical Qt boot setup and monkeypatching from QtPorting to a new QtInit module
* migrated the hydrus code for keyboardModifiers, event-pos, and globalPos all to the Qt6 equivalents so the monkeypatching is always going to be on older versions looking forward
* fiddled with QPoint and QPointF conversions a little so I _think_ Qt5 and Qt6 is always talking about the same type
* updated build scripts and requirements.txts for the new situation
* updated the help a bit for the new situation
## [Version 493](https://github.com/hydrusnetwork/hydrus/releases/tag/v493)
### EXIF
@ -262,30 +296,3 @@
* created a new database module for some rich file metadata and refactored some file filtering, history, and status testing code to it
* created new database module for file searching and moved all tag-based file searching code to it
* moved several other misc methods down to database modules
## [Version 484](https://github.com/hydrusnetwork/hydrus/releases/tag/v484)
### misc
* fixed the simple delete files dialog for trashed files. due to a logical oversight, the simple version was not testing 'trashed' status and so didn't see anything to permanently delete and would immediately dump out. now it shows the option for trashed files again, and if the selection includes trash and non-trash, it shows multiple options
* fixed an error in the 'show next pair' logic of the new duplicate filter queue where if it needed to auto-skip through the end of the current batch and load up the next batch (issues #1139, #1143)
* a new setting on _options->media_ now lets you set the scanbar to be small and simple instead of hidden when the mouse is moved away. I liked this so much personally it is now the default for new users. try it out!
* the media viewer's taglist hover window will now never send a mouse wheel event up to the media viewer canvas (so scrolling the tags won't accidentally do previous/next if you hit the end of the list scrollbar)
* I think I have fixed the bug where going on the media viewer from borderless fullscreen to a regular window would not trigger a media container resize if the media perfectly fitted the ratio of the fullscreen monitor!
* the system tray icon now has minimise/restore entries
* to reduce confusion, when a content parser vetoes, it now prepends the file import 'note' with 'veto: '
* the 'clear service info cache' job under _database->regenerate_ is renamed to 'service info numbers' and now has a service selector so you can, let's say, regen your miscounted 'number of files in trash' count without triggering a complete recount of every single mapping on the PTR the next time you open review services
* hydrus now recognises most (and maybe all) windows executables so it can discard them from imports confidently. a user discovered an interesting exe with embedded audio that ffmpeg was seeing as an mp3--this no longer occurs
* the 'edit string conversion step' dialog now saves a new default (which is used on 'add' events) every time you ok it. 'append extra text' is no longer the universal default!
* the 'edit tag rule' dialog in the parsing system now starts with the tag name field focused
* updated 'getting started/installing' help to talk more about mpv on Linux. the 'libgmodule' problem _seems_ to have a solid fix now, which is properly written out there. thanks to the users who figured all this out and provided feedback
### multiple local file services
* the media viewer menu now offers add/move actions just like the thumb grid
* added a new shortcut action that lets you specify add/move jobs. it is available in the media shortcut set and will work in the thumbnail grid and the media viewer
* add/move is now nicer in edge cases. files are filtered better to ensure only local media files end up in a job (e.g. if you were to try to move files out of the repository update domain using a shortcut), and 'add' commands from trashed files are naturally and silently converted to a pure undelete
### boring code cleanup
* refactored the UI side of multiple local file services add/move commands. various functions to select, filter, and question the user on actions are now pulled to a separate simple module where other parts of the UI can also access them, and there is now just one isolated pipeline for file service add/move content updates.
* if a 'move' job is started without a source service and multiple services could apply, the main routine will now ask the user which to use using a selector that shows how many files each choice will affect
* also rewrote the add/move menu population code, fixed a couple little issues, and refactored it to a module the media viewer canvas can use
* wrote a new menu builder that can place a list of items either as a single item (if the list is length 1), or make a submenu if there are more. it drives the new add/move commands and now the behind the scenes of all other service-based menu population

34
docs/old_changelog.html
View File

@ -33,6 +33,40 @@
<div class="content">
<h3 id="changelog"><a href="#changelog">changelog</a></h3>
<ul>
<li><h3 id="version_494"><a href="#version_494">version 494</a></h3></li>
<ul>
<li>QT6:</li>
<li>thanks to a user's help, we are rolling out a Qt6 test build this week. we've been running Qt5 for a few years now. 6 is mostly a very large bugfix patch, and I am hopeful this update will relieve several legacy issues related to UI scale, colour support, draw flickering, and other unusual stuff. so far, it is working for me great. I'll be putting out joint 5 and 6 builds for 4-8 weeks, to iron out any big problems, and then I'll switch over to 6 releases exclusively. if you are an advanced user, please give it a go this or next week and let me know if you run into any traceback errors about deprecated method names or completely jank layout in the less used parts of the program</li>
<li>the actual changes you'll see are mostly style, just slightly different font spacing, things like that. if you have a system-baked Qt5 style that hydrus magically inherits, this will no longer work, you need to get a Qt6 version of the style (although I understand this is happening already for the popular styles, so you may already have them)</li>
<li>users on Windows 7 and similarly old OS versions are unable to run Qt6 programs, sorry!</li>
<li>I intend to keep the code 5-compatible, and users who run from source can choose whichever version of Qt they prefer, as here in the help: https://hydrusnetwork.github.io/hydrus/running_from_source.html#qt</li>
<li>the linux Qt6 build also goes up from ubuntu 18.04 to 20.04. let me know if you have any trouble, but it feels like it is time to update this too</li>
<li>.</li>
<li>file import options overhaul:</li>
<li>I wanted to do note parsing this week, but when I reviewed the whole job, there wasn't enough time to do it properly. so, in prep for a cleaner introduction of 'note import options' next week, I am overhauling how the other import options do some stuff</li>
<li>all file import options now support filetype filtering! it uses the same control as system:filetype or in import folders, but with some improved logic. on update, existing import folder filetype settings will be copied down to the file import options</li>
<li>file import options now work on a similar 'default' system as tag import options. existing file import options will stay as-is, but new ones will begin in a 'use the default settings at time of import' state. those defaults are editable under _options->importing_. for now I am not adding a 'use this file import options default for this web domain' system, but it might happen in future. let's see how this all shakes out first</li>
<li>the file import options button now has a right-click menu like the tag import options button</li>
<li>the manage subscriptions panel now has a 'overwrite file import options' button to mass-set FIO</li>
<li>cleaned up a bunch of old file import and import options code</li>
<li>.</li>
<li>misc:</li>
<li>system:filetype now remembers meta filetypes better. if you select 'all video', it will now still select all video even if hydev adds support for a new video type in future. also if you select 'video + animations', it'll say that rather than listing out every possible specific-type</li>
<li>fixed an issue where loading a favourite search wasn't always setting 'include current/pending' values on the buttons correct</li>
<li>fixed up a status display in the gallery downloader and watcher pages--if you pause an importer while it is doing work, it now says 'pausing...' as its status until any current jobs are finished. it was giving empty text before, as if it were finished already</li>
<li>fixed some unusual behaviour with downloader highlighting where the first query pended to an empty page was secretly highlighted for the next session load, and fixed the 'subscription gap downloader' also doing this and not obeying the normal 'highlight new downloaders if nothing already highlighted' option</li>
<li>improved the error when the 'make sure this directory exists' function runs into a file with that pathname</li>
<li>fixed a rare selection position error, maybe Qt6 only, when clicking in the thumbnail grid as it is loading</li>
<li>.</li>
<li>boring Qt6 code cleanup:</li>
<li>as a side thing, I set up quick-launch environments for QtPy5, QtPy6, PySide2, and PySide6 in my IDE this week, so I can now test all these situations and jump back in time no problem in future</li>
<li>integrated a user's patch to bring us up to Qt6 compatibility and did a little more work to get it backwards compatible with older qtpy and Qt5</li>
<li>refactored the critical Qt boot setup and monkeypatching from QtPorting to a new QtInit module</li>
<li>migrated the hydrus code for keyboardModifiers, event-pos, and globalPos all to the Qt6 equivalents so the monkeypatching is always going to be on older versions looking forward</li>
<li>fiddled with QPoint and QPointF conversions a little so I _think_ Qt5 and Qt6 is always talking about the same type</li>
<li>updated build scripts and requirements.txts for the new situation</li>
<li>updated the help a bit for the new situation</li>
</ul>
<li><h3 id="version_493"><a href="#version_493">version 493</a></h3></li>
<ul>
<li>EXIF:</li>

4
hydrus/client/ClientController.py
View File

@ -1963,7 +1963,7 @@ class Controller( HydrusController.HydrusController ):
self._ShutdownManagers()
self.frame_splash_status.SetText( 'waiting for daemons to exit' )
self.frame_splash_status.SetText( 'waiting for workers to exit' )
self._ShutdownDaemons()
@ -1989,7 +1989,7 @@ class Controller( HydrusController.HydrusController ):
try:
self.frame_splash_status.SetText( 'waiting for twisted to exit' )
self.frame_splash_status.SetText( 'waiting for services to exit' )
self.SetRunningTwistedServices( [] )

24
hydrus/client/ClientOptions.py
View File

@ -12,6 +12,7 @@ from hydrus.core import HydrusTags
from hydrus.client import ClientConstants as CC
from hydrus.client import ClientDefaults
from hydrus.client import ClientDuplicates
from hydrus.client.importing.options import FileImportOptions
class ClientOptions( HydrusSerialisable.SerialisableBase ):
@ -966,7 +967,17 @@ class ClientOptions( HydrusSerialisable.SerialisableBase ):
with self._lock:
return self._dictionary[ 'default_file_import_options' ][ options_type ]
if options_type == FileImportOptions.IMPORT_TYPE_LOUD:
key = 'loud'
else:
key = 'quiet'
return self._dictionary[ 'default_file_import_options' ][ key ]
@ -1411,7 +1422,16 @@ class ClientOptions( HydrusSerialisable.SerialisableBase ):
with self._lock:
self._dictionary[ 'default_file_import_options' ][ options_type ] = file_import_options
if options_type == FileImportOptions.IMPORT_TYPE_LOUD:
key = 'loud'
else:
key = 'quiet'
self._dictionary[ 'default_file_import_options' ][ key ] = file_import_options

132
hydrus/client/ClientSearch.py
View File

@ -108,6 +108,32 @@ SYSTEM_PREDICATE_TYPES = {
IGNORED_TAG_SEARCH_CHARACTERS = '[](){}/\\"\'-_'
IGNORED_TAG_SEARCH_CHARACTERS_UNICODE_TRANSLATE = { ord( char ) : ' ' for char in IGNORED_TAG_SEARCH_CHARACTERS }
def ConvertSpecificFiletypesToSummary( specific_mimes: typing.Collection[ int ], only_searchable = True ) -> typing.Collection[ int ]:
specific_mimes_to_process = set( specific_mimes )
summary_mimes = set()
for ( general_mime, mime_group ) in HC.general_mimetypes_to_mime_groups.items():
if only_searchable:
mime_group = set( mime_group )
mime_group.intersection_update( HC.SEARCHABLE_MIMES )
if specific_mimes_to_process.issuperset( mime_group ):
summary_mimes.add( general_mime )
specific_mimes_to_process.difference_update( mime_group )
summary_mimes.update( specific_mimes_to_process )
return summary_mimes
def ConvertSubtagToSearchable( subtag ):
if subtag == '':
@ -125,6 +151,47 @@ def ConvertSubtagToSearchable( subtag ):
return subtag
def ConvertSummaryFiletypesToSpecific( summary_mimes: typing.Collection[ int ], only_searchable = True ) -> typing.Collection[ int ]:
specific_mimes = set()
for mime in summary_mimes:
if mime in HC.GENERAL_FILETYPES:
specific_mimes.update( HC.general_mimetypes_to_mime_groups[ mime ] )
else:
specific_mimes.add( mime )
if only_searchable:
specific_mimes.intersection_update( HC.SEARCHABLE_MIMES )
return specific_mimes
def ConvertSummaryFiletypesToString( summary_mimes: typing.Collection[ int ] ) -> str:
if set( summary_mimes ) == HC.GENERAL_FILETYPES:
mime_text = 'anything'
else:
summary_mimes = sorted( summary_mimes, key = lambda m: HC.mime_mimetype_string_lookup[ m ] )
mime_text = ', '.join( [ HC.mime_string_lookup[ mime ] for mime in summary_mimes ] )
return mime_text
def ConvertTagToSearchable( tag ):
( namespace, subtag ) = HydrusTags.SplitTag( tag )
@ -442,11 +509,14 @@ class FileSystemPredicates( object ):
if predicate_type == PREDICATE_TYPE_SYSTEM_MIME:
mimes = value
summary_mimes = value
if isinstance( mimes, int ): mimes = ( mimes, )
if isinstance( summary_mimes, int ):
summary_mimes = ( summary_mimes, )
self._common_info[ 'mimes' ] = mimes
self._common_info[ 'mimes' ] = ConvertSummaryFiletypesToSpecific( summary_mimes )
if predicate_type == PREDICATE_TYPE_SYSTEM_DURATION:
@ -1446,7 +1516,7 @@ class Predicate( HydrusSerialisable.SerialisableBase ):
SERIALISABLE_TYPE = HydrusSerialisable.SERIALISABLE_TYPE_PREDICATE
SERIALISABLE_NAME = 'File Search Predicate'
SERIALISABLE_VERSION = 4
SERIALISABLE_VERSION = 5
def __init__(
self,
@ -1456,6 +1526,11 @@ class Predicate( HydrusSerialisable.SerialisableBase ):
count = None
):
if predicate_type == PREDICATE_TYPE_SYSTEM_MIME and value is not None:
value = tuple( ConvertSpecificFiletypesToSummary( value ) )
if isinstance( value, ( list, set ) ):
value = tuple( value )
@ -1713,6 +1788,24 @@ class Predicate( HydrusSerialisable.SerialisableBase ):
return ( 4, new_serialisable_info )
if version == 4:
( predicate_type, serialisable_value, inclusive ) = old_serialisable_info
if predicate_type == PREDICATE_TYPE_SYSTEM_MIME:
specific_mimes = serialisable_value
summary_mimes = ConvertSpecificFiletypesToSummary( specific_mimes )
serialisable_value = tuple( summary_mimes )
new_serialisable_info = ( predicate_type, serialisable_value, inclusive )
return ( 5, new_serialisable_info )
def GetCopy( self ):
@ -2406,36 +2499,9 @@ class Predicate( HydrusSerialisable.SerialisableBase ):
if self._value is not None:
mimes = self._value
summary_mimes = self._value
if set( mimes ) == set( HC.SEARCHABLE_MIMES ):
mime_text = 'anything'
elif set( mimes ) == set( HC.SEARCHABLE_MIMES ).intersection( set( HC.APPLICATIONS ) ):
mime_text = 'application'
elif set( mimes ) == set( HC.SEARCHABLE_MIMES ).intersection( set( HC.AUDIO ) ):
mime_text = 'audio'
elif set( mimes ) == set( HC.SEARCHABLE_MIMES ).intersection( set( HC.IMAGES ) ):
mime_text = 'image'
elif set( mimes ) == set( HC.SEARCHABLE_MIMES ).intersection( set( HC.ANIMATIONS ) ):
mime_text = 'animation'
elif set( mimes ) == set( HC.SEARCHABLE_MIMES ).intersection( set( HC.VIDEO ) ):
mime_text = 'video'
else:
mime_text = ', '.join( [ HC.mime_string_lookup[ mime ] for mime in mimes ] )
mime_text = ConvertSummaryFiletypesToString( summary_mimes )
base += ' is ' + mime_text

21
hydrus/client/gui/ClientGUI.py
View File

@ -710,7 +710,7 @@ class FrameGUI( ClientGUITopLevelWindows.MainFrameThatResizes, CAC.ApplicationCo
elif QtInit.WE_ARE_PYQT:
from PyQt5.Qt import PYQT_VERSION_STR # pylint: disable=E0401,E0611
from PyQt5.sip import SIP_VERSION_STR # pylint: disable=E0401
from PyQt5.sip import SIP_VERSION_STR # pylint: disable=E0401,E0611
library_versions.append( ( 'PyQt5', PYQT_VERSION_STR ) )
library_versions.append( ( 'sip', SIP_VERSION_STR ) )
@ -728,8 +728,8 @@ class FrameGUI( ClientGUITopLevelWindows.MainFrameThatResizes, CAC.ApplicationCo
elif QtInit.WE_ARE_PYQT:
from PyQt6.QtCore import PYQT_VERSION_STR # pylint: disable=E0401
from PyQt6.sip import SIP_VERSION_STR # pylint: disable=E0401
from PyQt6.QtCore import PYQT_VERSION_STR # pylint: disable=E0401,E0611
from PyQt6.sip import SIP_VERSION_STR # pylint: disable=E0401,E0611
library_versions.append( ( 'PyQt6', PYQT_VERSION_STR ) )
library_versions.append( ( 'sip', SIP_VERSION_STR ) )
@ -1461,7 +1461,10 @@ class FrameGUI( ClientGUITopLevelWindows.MainFrameThatResizes, CAC.ApplicationCo
job_key.SetStatusTitle( 'sub gap downloader test' )
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'quiet' )
from hydrus.client.importing.options import FileImportOptions
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
from hydrus.client.importing.options import TagImportOptions
@ -3968,7 +3971,7 @@ class FrameGUI( ClientGUITopLevelWindows.MainFrameThatResizes, CAC.ApplicationCo
try:
job_key.SetVariable( 'popup_text_1', 'Waiting for import folders to finish.' )
job_key.SetVariable( 'popup_text_1', 'Waiting for export folders to finish.' )
controller.pub( 'message', job_key )
@ -4109,7 +4112,7 @@ class FrameGUI( ClientGUITopLevelWindows.MainFrameThatResizes, CAC.ApplicationCo
try:
controller.CallBlockingToQt(self, qt_do_it)
controller.CallBlockingToQt( self, qt_do_it )
except HydrusExceptions.QtDeadWindowException:
@ -6864,11 +6867,9 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
HydrusData.ShowText( 'Sorry, could not create the downloader page! Is your session super full atm?' )
management_controller = page.GetManagementController()
panel = page.GetManagementPanel()
multiple_gallery_import = management_controller.GetVariable( 'multiple_gallery_import' )
multiple_gallery_import.PendSubscriptionGapDownloader( gug_key_and_name, query_text, file_import_options, tag_import_options, file_limit )
panel.PendSubscriptionGapDownloader( gug_key_and_name, query_text, file_import_options, tag_import_options, file_limit )
self._notebook.ShowPage( page )

155
hydrus/client/gui/ClientGUIImport.py
View File

@ -101,22 +101,30 @@ class CheckerOptionsButton( ClientGUICommon.BetterButton ):
class FileImportOptionsButton( ClientGUICommon.BetterButton ):
def __init__( self, parent, file_import_options, show_downloader_options, update_callable = None ):
def __init__( self, parent, file_import_options, show_downloader_options, update_callable = None, allow_default_selection = True ):
ClientGUICommon.BetterButton.__init__( self, parent, 'file import options', self._EditOptions )
self._file_import_options = file_import_options
self._show_downloader_options = show_downloader_options
self._update_callable = update_callable
self._allow_default_selection = allow_default_selection
self._SetToolTip()
def _Copy( self ):
json_string = self._file_import_options.DumpToString()
HG.client_controller.pub( 'clipboard', 'text', json_string )
def _EditOptions( self ):
with ClientGUITopLevelWindowsPanels.DialogEdit( self, 'edit file import options' ) as dlg:
panel = ClientGUIScrolledPanelsEdit.EditFileImportOptions( dlg, self._file_import_options, self._show_downloader_options )
panel = ClientGUIScrolledPanelsEdit.EditFileImportOptions( dlg, self._file_import_options, self._show_downloader_options, self._allow_default_selection )
dlg.SetPanel( panel )
@ -129,6 +137,49 @@ class FileImportOptionsButton( ClientGUICommon.BetterButton ):
def _Paste( self ):
try:
raw_text = HG.client_controller.GetClipboardText()
except HydrusExceptions.DataMissing as e:
QW.QMessageBox.critical( self, 'Error', str(e) )
return
try:
file_import_options = HydrusSerialisable.CreateFromString( raw_text )
if not isinstance( file_import_options, FileImportOptions.FileImportOptions ):
raise Exception( 'Not a File Import Options!' )
except Exception as e:
QW.QMessageBox.critical( self, 'Error', 'I could not understand what was in the clipboard' )
HydrusData.ShowException( e )
return
self._SetValue( file_import_options )
def _SetDefault( self ):
file_import_options = self._file_import_options.Duplicate()
file_import_options.SetIsDefault( True )
self._SetValue( file_import_options )
def _SetToolTip( self ):
self.setToolTip( self._file_import_options.GetSummary() )
@ -146,16 +197,56 @@ class FileImportOptionsButton( ClientGUICommon.BetterButton ):
def contextMenuEvent( self, event ):
if event.reason() == QG.QContextMenuEvent.Keyboard:
self.ShowMenu()
def GetValue( self ):
return self._file_import_options
def mouseReleaseEvent( self, event ):
if event.button() != QC.Qt.RightButton:
ClientGUICommon.BetterButton.mouseReleaseEvent( self, event )
return
self.ShowMenu()
def SetValue( self, file_import_options ):
self._SetValue( file_import_options )
def ShowMenu( self ):
menu = QW.QMenu()
ClientGUIMenus.AppendMenuItem( menu, 'copy to clipboard', 'Serialise this file import options and copy it to clipboard.', self._Copy )
ClientGUIMenus.AppendSeparator( menu )
ClientGUIMenus.AppendMenuItem( menu, 'paste from clipboard', 'Try to import serialised file import options from the clipboard.', self._Paste )
if not self._file_import_options.IsDefault():
ClientGUIMenus.AppendSeparator( menu )
ClientGUIMenus.AppendMenuItem( menu, 'set to default', 'Set this file import options to defer to the defaults.', self._SetDefault )
CGC.core().PopupMenu( self, menu )
class FilenameTaggingOptionsPanel( QW.QWidget ):
movePageLeft = QC.Signal()
@ -990,7 +1081,7 @@ class EditImportFolderPanel( ClientGUIScrolledPanels.EditPanel ):
self._import_folder = import_folder
( name, path, mimes, file_import_options, tag_import_options, tag_service_keys_to_filename_tagging_options, actions, action_locations, period, check_regularly, paused, check_now, show_working_popup, publish_files_to_popup_button, publish_files_to_page ) = self._import_folder.ToTuple()
( name, path, file_import_options, tag_import_options, tag_service_keys_to_filename_tagging_options, actions, action_locations, period, check_regularly, paused, check_now, show_working_popup, publish_files_to_popup_button, publish_files_to_page ) = self._import_folder.ToTuple()
self._folder_box = ClientGUICommon.StaticBox( self, 'folder options' )
@ -1016,8 +1107,6 @@ class EditImportFolderPanel( ClientGUIScrolledPanels.EditPanel ):
self._file_box = ClientGUICommon.StaticBox( self, 'file options' )
self._mimes = ClientGUIOptionsPanels.OptionsPanelMimes( self._file_box, HC.ALLOWED_MIMES )
def create_choice():
choice = ClientGUICommon.BetterChoice( self._file_box )
@ -1082,8 +1171,6 @@ class EditImportFolderPanel( ClientGUIScrolledPanels.EditPanel ):
self._publish_files_to_popup_button.setChecked( publish_files_to_popup_button )
self._publish_files_to_page.setChecked( publish_files_to_page )
self._mimes.SetValue( mimes )
self._action_successful.SetValue( actions[ CC.STATUS_SUCCESSFUL_AND_NEW ] )
if CC.STATUS_SUCCESSFUL_AND_NEW in action_locations:
@ -1135,12 +1222,6 @@ class EditImportFolderPanel( ClientGUIScrolledPanels.EditPanel ):
#
rows = []
rows.append( ( 'file types to import: ', self._mimes ) )
mimes_gridbox = ClientGUICommon.WrapInGrid( self._file_box, rows, expand_text = True )
gridbox = QP.GridLayout( cols = 3 )
gridbox.setColumnStretch( 1, 1 )
@ -1161,7 +1242,6 @@ class EditImportFolderPanel( ClientGUIScrolledPanels.EditPanel ):
QP.AddToLayout( gridbox, self._action_failed, CC.FLAGS_EXPAND_BOTH_WAYS )
QP.AddToLayout( gridbox, self._location_failed, CC.FLAGS_EXPAND_BOTH_WAYS )
self._file_box.Add( mimes_gridbox, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
self._file_box.Add( gridbox, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
self._file_box.Add( self._file_import_options, CC.FLAGS_EXPAND_PERPENDICULAR )
@ -1413,7 +1493,6 @@ class EditImportFolderPanel( ClientGUIScrolledPanels.EditPanel ):
name = self._name.text()
path = self._path.GetPath()
mimes = self._mimes.GetValue()
file_import_options = self._file_import_options.GetValue()
tag_import_options = self._tag_import_options.GetValue()
@ -1457,7 +1536,7 @@ class EditImportFolderPanel( ClientGUIScrolledPanels.EditPanel ):
tag_service_keys_to_filename_tagging_options = dict( self._filename_tagging_options.GetData() )
self._import_folder.SetTuple( name, path, mimes, file_import_options, tag_import_options, tag_service_keys_to_filename_tagging_options, actions, action_locations, period, check_regularly, paused, check_now, show_working_popup, publish_files_to_popup_button, publish_files_to_page )
self._import_folder.SetTuple( name, path, file_import_options, tag_import_options, tag_service_keys_to_filename_tagging_options, actions, action_locations, period, check_regularly, paused, check_now, show_working_popup, publish_files_to_popup_button, publish_files_to_page )
return self._import_folder
@ -2202,7 +2281,7 @@ class TagImportOptionsButton( ClientGUICommon.BetterButton ):
with ClientGUITopLevelWindowsPanels.DialogEdit( self, 'edit tag import options' ) as dlg:
panel = ClientGUIScrolledPanelsEdit.EditTagImportOptionsPanel( dlg, self._tag_import_options, self._show_downloader_options, allow_default_selection = self._allow_default_selection )
panel = ClientGUIScrolledPanelsEdit.EditTagImportOptionsPanel( dlg, self._tag_import_options, self._show_downloader_options, self._allow_default_selection )
dlg.SetPanel( panel )
@ -2237,19 +2316,25 @@ class TagImportOptionsButton( ClientGUICommon.BetterButton ):
raise Exception( 'Not a Tag Import Options!' )
self._tag_import_options = tag_import_options
except Exception as e:
QW.QMessageBox.critical( self, 'Error', 'I could not understand what was in the clipboard' )
HydrusData.ShowException( e )
return
self._SetValue( tag_import_options )
def _SetDefault( self ):
self._tag_import_options.SetDefault()
tag_import_options = self._tag_import_options.Duplicate()
tag_import_options.SetIsDefault( True )
self._SetValue( tag_import_options )
def _SetToolTip( self ):
@ -2279,6 +2364,11 @@ class TagImportOptionsButton( ClientGUICommon.BetterButton ):
def GetValue( self ):
return self._tag_import_options
def mouseReleaseEvent( self, event ):
if event.button() != QC.Qt.RightButton:
@ -2291,6 +2381,16 @@ class TagImportOptionsButton( ClientGUICommon.BetterButton ):
self.ShowMenu()
def SetNamespaces( self, namespaces ):
self._namespaces = namespaces
def SetValue( self, tag_import_options ):
self._SetValue( tag_import_options )
def ShowMenu( self ):
menu = QW.QMenu()
@ -2311,21 +2411,6 @@ class TagImportOptionsButton( ClientGUICommon.BetterButton ):
CGC.core().PopupMenu( self, menu )
def GetValue( self ):
return self._tag_import_options
def SetNamespaces( self, namespaces ):
self._namespaces = namespaces
def SetValue( self, tag_import_options ):
self._SetValue( tag_import_options )
class WatcherReviewPanel( ClientGUICommon.StaticBox ):
def __init__( self, parent, page_key, name = 'watcher' ):

4
hydrus/client/gui/ClientGUILogin.py
View File

@ -1647,10 +1647,10 @@ class EditLoginScriptPanel( ClientGUIScrolledPanels.EditPanel ):
def do_it( login_script, domain, credentials, network_job_presentation_context_factory ):
login_result = 'login did not finish'
try:
login_result = 'login did not finish'
# a potential here is to properly inform the login manager of the domain map and hence read back the invalidation text
# but I am catching the info in the raised exception, so nbd really, I think

3
hydrus/client/gui/ClientGUIOptionsPanels.py
View File

@ -4,6 +4,7 @@ from qtpy import QtWidgets as QW
from hydrus.core import HydrusConstants as HC
from hydrus.client import ClientConstants as CC
from hydrus.client import ClientSearch
from hydrus.client.gui import ClientGUIFunctions
from hydrus.client.gui import QtPorting as QP
from hydrus.client.gui.widgets import ClientGUICommon
@ -217,6 +218,8 @@ class OptionsPanelMimes( OptionsPanel ):
def SetValue( self, checked_mimes ):
checked_mimes = ClientSearch.ConvertSummaryFiletypesToSpecific( checked_mimes, only_searchable = False )
for ( mime, checkbox ) in self._mimes_to_checkboxes.items():
if mime in checked_mimes:

8
hydrus/client/gui/ClientGUIPanels.py
View File

@ -73,6 +73,9 @@ class IPFSDaemonStatusAndInteractionPanel( ClientGUICommon.StaticBox ):
def do_it( service ):
result = ''
nocopy_available = False
try:
nocopy_available = service.GetNoCopyAvailable()
@ -132,6 +135,9 @@ class IPFSDaemonStatusAndInteractionPanel( ClientGUICommon.StaticBox ):
def do_it( service ):
result = ''
is_running = False
try:
version = service.GetDaemonVersion()
@ -186,6 +192,8 @@ class IPFSDaemonStatusAndInteractionPanel( ClientGUICommon.StaticBox ):
def do_it( service ):
success = False
try:
success = service.EnableNoCopy( True )

243
hydrus/client/gui/ClientGUIScrolledPanelsEdit.py
View File

@ -19,6 +19,7 @@ from hydrus.client.gui import ClientGUIDialogs
from hydrus.client.gui import ClientGUIDialogsQuick
from hydrus.client.gui import ClientGUIFunctions
from hydrus.client.gui import ClientGUIImport
from hydrus.client.gui import ClientGUIOptionsPanels
from hydrus.client.gui import ClientGUIScrolledPanels
from hydrus.client.gui import ClientGUIShortcuts
from hydrus.client.gui import ClientGUITags
@ -1423,7 +1424,7 @@ class EditDuplicateActionOptionsPanel( ClientGUIScrolledPanels.EditPanel ):
class EditFileImportOptions( ClientGUIScrolledPanels.EditPanel ):
def __init__( self, parent: QW.QWidget, file_import_options: FileImportOptions.FileImportOptions, show_downloader_options: bool ):
def __init__( self, parent: QW.QWidget, file_import_options: FileImportOptions.FileImportOptions, show_downloader_options: bool, allow_default_selection: bool ):
ClientGUIScrolledPanels.EditPanel.__init__( self, parent )
@ -1433,7 +1434,32 @@ class EditFileImportOptions( ClientGUIScrolledPanels.EditPanel ):
#
pre_import_panel = ClientGUICommon.StaticBox( self, 'pre-import checks' )
default_panel = ClientGUICommon.StaticBox( self, 'default options' )
self._use_default_dropdown = ClientGUICommon.BetterChoice( default_panel )
self._use_default_dropdown.addItem( 'use the default file import options at the time of import', True )
self._use_default_dropdown.addItem( 'set custom file import options just for this downloader', False )
tt = 'Normally, the client will refer to the defaults (as set under "options->importing") at the time of import.'
tt += os.linesep * 2
tt += 'It is easier to work this way, since you can change a single default setting and update all current and future downloaders that refer to those defaults, whereas having specific options for every subscription or downloader means you have to update every single one just to make a little change somewhere.'
tt += os.linesep * 2
tt += 'But if you are doing a one-time import that has some unusual file rules, set them here.'
self._use_default_dropdown.setToolTip( tt )
#
self._load_default_options = ClientGUICommon.BetterButton( self, 'load one of the default options', self._LoadDefaultOptions )
#
self._specific_options_panel = QW.QWidget( self )
#
pre_import_panel = ClientGUICommon.StaticBox( self._specific_options_panel, 'pre-import checks' )
self._exclude_deleted = QW.QCheckBox( pre_import_panel )
@ -1461,6 +1487,8 @@ class EditFileImportOptions( ClientGUIScrolledPanels.EditPanel ):
self._allow_decompression_bombs.setToolTip( tt )
self._mimes = ClientGUIOptionsPanels.OptionsPanelMimes( pre_import_panel, HC.ALLOWED_MIMES )
self._min_size = ClientGUIControls.NoneableBytesControl( pre_import_panel )
self._min_size.SetValue( 5 * 1024 )
@ -1487,7 +1515,11 @@ class EditFileImportOptions( ClientGUIScrolledPanels.EditPanel ):
#
destination_panel = ClientGUICommon.StaticBox( self, 'import destinations' )
self._use_default_dropdown.SetValue( file_import_options.IsDefault() )
#
destination_panel = ClientGUICommon.StaticBox( self._specific_options_panel, 'import destinations' )
self._destination_location_context_st = ClientGUICommon.BetterStaticText( destination_panel, 'If you have more than one local file service, you can send these imports to other/multiple locations.' )
@ -1501,7 +1533,7 @@ class EditFileImportOptions( ClientGUIScrolledPanels.EditPanel ):
#
post_import_panel = ClientGUICommon.StaticBox( self, 'post-import actions' )
post_import_panel = ClientGUICommon.StaticBox( self._specific_options_panel, 'post-import actions' )
self._auto_archive = QW.QCheckBox( post_import_panel )
self._associate_primary_urls = QW.QCheckBox( post_import_panel )
@ -1521,7 +1553,7 @@ class EditFileImportOptions( ClientGUIScrolledPanels.EditPanel ):
#
presentation_static_box = ClientGUICommon.StaticBox( self, 'presentation options' )
presentation_static_box = ClientGUICommon.StaticBox( self._specific_options_panel, 'presentation options' )
presentation_import_options = file_import_options.GetPresentationImportOptions()
@ -1529,27 +1561,16 @@ class EditFileImportOptions( ClientGUIScrolledPanels.EditPanel ):
#
( exclude_deleted, do_not_check_known_urls_before_importing, do_not_check_hashes_before_importing, allow_decompression_bombs, min_size, max_size, max_gif_size, min_resolution, max_resolution ) = file_import_options.GetPreImportOptions()
self._exclude_deleted.setChecked( exclude_deleted )
self._do_not_check_known_urls_before_importing.setChecked( do_not_check_known_urls_before_importing )
self._do_not_check_hashes_before_importing.setChecked( do_not_check_hashes_before_importing )
self._allow_decompression_bombs.setChecked( allow_decompression_bombs )
self._min_size.SetValue( min_size )
self._max_size.SetValue( max_size )
self._max_gif_size.SetValue( max_gif_size )
self._min_resolution.SetValue( min_resolution )
self._max_resolution.SetValue( max_resolution )
self._SetValue( file_import_options )
#
automatic_archive = file_import_options.AutomaticallyArchives()
associate_primary_urls = file_import_options.ShouldAssociatePrimaryURLs()
associate_source_urls = file_import_options.ShouldAssociateSourceURLs()
default_panel.Add( self._use_default_dropdown, CC.FLAGS_EXPAND_PERPENDICULAR )
self._auto_archive.setChecked( automatic_archive )
self._associate_primary_urls.setChecked( associate_primary_urls )
self._associate_source_urls.setChecked( associate_source_urls )
if not allow_default_selection:
default_panel.hide()
#
@ -1568,6 +1589,7 @@ class EditFileImportOptions( ClientGUIScrolledPanels.EditPanel ):
self._do_not_check_hashes_before_importing.setVisible( False )
rows.append( ( 'allowed filetypes: ', self._mimes ) )
rows.append( ( 'allow decompression bombs: ', self._allow_decompression_bombs ) )
rows.append( ( 'minimum filesize: ', self._min_size ) )
rows.append( ( 'maximum filesize: ', self._max_size ) )
@ -1611,13 +1633,23 @@ class EditFileImportOptions( ClientGUIScrolledPanels.EditPanel ):
#
specific_vbox = QP.VBoxLayout()
QP.AddToLayout( specific_vbox, pre_import_panel, CC.FLAGS_EXPAND_PERPENDICULAR )
QP.AddToLayout( specific_vbox, destination_panel, CC.FLAGS_EXPAND_PERPENDICULAR )
QP.AddToLayout( specific_vbox, post_import_panel, CC.FLAGS_EXPAND_PERPENDICULAR )
QP.AddToLayout( specific_vbox, presentation_static_box, CC.FLAGS_EXPAND_PERPENDICULAR )
self._specific_options_panel.setLayout( specific_vbox )
#
vbox = QP.VBoxLayout()
QP.AddToLayout( vbox, help_hbox, CC.FLAGS_ON_RIGHT )
QP.AddToLayout( vbox, pre_import_panel, CC.FLAGS_EXPAND_PERPENDICULAR )
QP.AddToLayout( vbox, destination_panel, CC.FLAGS_EXPAND_PERPENDICULAR )
QP.AddToLayout( vbox, post_import_panel, CC.FLAGS_EXPAND_PERPENDICULAR )
QP.AddToLayout( vbox, presentation_static_box, CC.FLAGS_EXPAND_PERPENDICULAR )
QP.AddToLayout( vbox, default_panel, CC.FLAGS_EXPAND_PERPENDICULAR )
QP.AddToLayout( vbox, self._load_default_options, CC.FLAGS_EXPAND_PERPENDICULAR )
QP.AddToLayout( vbox, self._specific_options_panel, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
vbox.addStretch( 1 )
@ -1625,6 +1657,86 @@ class EditFileImportOptions( ClientGUIScrolledPanels.EditPanel ):
self._destination_location_context.locationChanged.connect( self._UpdateLocationText )
self._use_default_dropdown.currentIndexChanged.connect( self._UpdateIsDefault )
self._UpdateIsDefault()
def _LoadDefaultOptions( self ):
loud_file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( FileImportOptions.IMPORT_TYPE_LOUD )
quiet_file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( FileImportOptions.IMPORT_TYPE_QUIET )
choice_tuples = []
choice_tuples.append( ( 'loud default', loud_file_import_options ) )
choice_tuples.append( ( 'quiet default', quiet_file_import_options ) )
try:
default_file_import_options = ClientGUIDialogsQuick.SelectFromList( self, 'Select which default', choice_tuples, sort_tuples = False )
except HydrusExceptions.CancelledException:
return
if default_file_import_options is None:
return
self._SetValue( default_file_import_options )
def _SetValue( self, file_import_options: FileImportOptions.FileImportOptions ):
self._use_default_dropdown.SetValue( file_import_options.IsDefault() )
( exclude_deleted, do_not_check_known_urls_before_importing, do_not_check_hashes_before_importing, allow_decompression_bombs, min_size, max_size, max_gif_size, min_resolution, max_resolution ) = file_import_options.GetPreImportOptions()
mimes = file_import_options.GetAllowedSpecificFiletypes()
self._mimes.SetValue( mimes )
self._exclude_deleted.setChecked( exclude_deleted )
self._do_not_check_known_urls_before_importing.setChecked( do_not_check_known_urls_before_importing )
self._do_not_check_hashes_before_importing.setChecked( do_not_check_hashes_before_importing )
self._allow_decompression_bombs.setChecked( allow_decompression_bombs )
self._min_size.SetValue( min_size )
self._max_size.SetValue( max_size )
self._max_gif_size.SetValue( max_gif_size )
self._min_resolution.SetValue( min_resolution )
self._max_resolution.SetValue( max_resolution )
#
automatic_archive = file_import_options.AutomaticallyArchives()
associate_primary_urls = file_import_options.ShouldAssociatePrimaryURLs()
associate_source_urls = file_import_options.ShouldAssociateSourceURLs()
self._auto_archive.setChecked( automatic_archive )
self._associate_primary_urls.setChecked( associate_primary_urls )
self._associate_source_urls.setChecked( associate_source_urls )
#
destination_location_context = file_import_options.GetDestinationLocationContext()
destination_location_context.FixMissingServices( HG.client_controller.services_manager.FilterValidServiceKeys )
self._destination_location_context.SetValue( destination_location_context )
#
presentation_import_options = file_import_options.GetPresentationImportOptions()
self._presentation_import_options_edit_panel.SetValue( presentation_import_options )
#
self._UpdateIsDefault()
def _ShowHelp( self ):
@ -1653,6 +1765,22 @@ If you have a very large (10k+ files) file import page, consider hiding some or
QW.QMessageBox.information( self, 'Information', help_message )
def _UpdateIsDefault( self ):
is_default = self._use_default_dropdown.GetValue()
show_specific_options = not is_default
self._load_default_options.setVisible( show_specific_options )
self._specific_options_panel.setVisible( show_specific_options )
if not show_specific_options:
self.window().adjustSize()
def _UpdateLocationText( self ):
location_context = self._destination_location_context.GetValue()
@ -1675,30 +1803,40 @@ If you have a very large (10k+ files) file import page, consider hiding some or
def GetValue( self ) -> FileImportOptions.FileImportOptions:
exclude_deleted = self._exclude_deleted.isChecked()
do_not_check_known_urls_before_importing = self._do_not_check_known_urls_before_importing.isChecked()
do_not_check_hashes_before_importing = self._do_not_check_hashes_before_importing.isChecked()
allow_decompression_bombs = self._allow_decompression_bombs.isChecked()
min_size = self._min_size.GetValue()
max_size = self._max_size.GetValue()
max_gif_size = self._max_gif_size.GetValue()
min_resolution = self._min_resolution.GetValue()
max_resolution = self._max_resolution.GetValue()
automatic_archive = self._auto_archive.isChecked()
associate_primary_urls = self._associate_primary_urls.isChecked()
associate_source_urls = self._associate_source_urls.isChecked()
presentation_import_options = self._presentation_import_options_edit_panel.GetValue()
is_default = self._use_default_dropdown.GetValue()
file_import_options = FileImportOptions.FileImportOptions()
destination_location_context = self._destination_location_context.GetValue()
file_import_options.SetPreImportOptions( exclude_deleted, do_not_check_known_urls_before_importing, do_not_check_hashes_before_importing, allow_decompression_bombs, min_size, max_size, max_gif_size, min_resolution, max_resolution )
file_import_options.SetDestinationLocationContext( destination_location_context )
file_import_options.SetPostImportOptions( automatic_archive, associate_primary_urls, associate_source_urls )
file_import_options.SetPresentationImportOptions( presentation_import_options )
if is_default:
file_import_options.SetIsDefault( True )
else:
exclude_deleted = self._exclude_deleted.isChecked()
do_not_check_known_urls_before_importing = self._do_not_check_known_urls_before_importing.isChecked()
do_not_check_hashes_before_importing = self._do_not_check_hashes_before_importing.isChecked()
allow_decompression_bombs = self._allow_decompression_bombs.isChecked()
min_size = self._min_size.GetValue()
max_size = self._max_size.GetValue()
max_gif_size = self._max_gif_size.GetValue()
min_resolution = self._min_resolution.GetValue()
max_resolution = self._max_resolution.GetValue()
automatic_archive = self._auto_archive.isChecked()
associate_primary_urls = self._associate_primary_urls.isChecked()
associate_source_urls = self._associate_source_urls.isChecked()
presentation_import_options = self._presentation_import_options_edit_panel.GetValue()
destination_location_context = self._destination_location_context.GetValue()
file_import_options.SetPreImportOptions( exclude_deleted, do_not_check_known_urls_before_importing, do_not_check_hashes_before_importing, allow_decompression_bombs, min_size, max_size, max_gif_size, min_resolution, max_resolution )
file_import_options.SetAllowedSpecificFiletypes( self._mimes.GetValue() )
file_import_options.SetDestinationLocationContext( destination_location_context )
file_import_options.SetPostImportOptions( automatic_archive, associate_primary_urls, associate_source_urls )
file_import_options.SetPresentationImportOptions( presentation_import_options )
return file_import_options
@ -2501,6 +2639,13 @@ class EditPresentationImportOptions( ClientGUIScrolledPanels.EditPanel ):
return presentation_import_options
def SetValue( self, presentation_import_options: PresentationImportOptions.PresentationImportOptions ):
self._presentation_location.SetValue( presentation_import_options.GetLocationContext() )
self._presentation_status.SetValue( presentation_import_options.GetPresentationStatus() )
self._presentation_inbox.SetValue( presentation_import_options.GetPresentationInbox() )
class EditRegexFavourites( ClientGUIScrolledPanels.EditPanel ):
def __init__( self, parent: QW.QWidget, regex_favourites ):
@ -2625,7 +2770,7 @@ class EditRegexFavourites( ClientGUIScrolledPanels.EditPanel ):
class EditTagImportOptionsPanel( ClientGUIScrolledPanels.EditPanel ):
def __init__( self, parent: QW.QWidget, tag_import_options: TagImportOptions.TagImportOptions, show_downloader_options: bool, allow_default_selection: bool = False ):
def __init__( self, parent: QW.QWidget, tag_import_options: TagImportOptions.TagImportOptions, show_downloader_options: bool, allow_default_selection: bool ):
ClientGUIScrolledPanels.EditPanel.__init__( self, parent )
@ -2651,7 +2796,7 @@ class EditTagImportOptionsPanel( ClientGUIScrolledPanels.EditPanel ):
tt += os.linesep * 2
tt += 'It is easier to work this way, since you can change a single default setting and update all current and future downloaders that refer to those defaults, whereas having specific options for every subscription or downloader means you have to update every single one just to make a little change somewhere.'
tt += os.linesep * 2
tt += 'But if you are doing a one-time import that has some unusual tag rules, set some specific rules here.'
tt += 'But if you are doing a one-time import that has some unusual tag rules, set them here.'
self._use_default_dropdown.setToolTip( tt )

17
hydrus/client/gui/ClientGUIScrolledPanelsManagement.py
View File

@ -21,6 +21,7 @@ from hydrus.core import HydrusText
from hydrus.client import ClientApplicationCommand as CAC
from hydrus.client import ClientConstants as CC
from hydrus.client import ClientLocation
from hydrus.client.importing.options import FileImportOptions
from hydrus.client.gui import ClientGUIDialogs
from hydrus.client.gui import ClientGUIDialogsQuick
from hydrus.client.gui import ClientGUIFunctions
@ -1681,20 +1682,20 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
show_downloader_options = True
quiet_file_import_options = self._new_options.GetDefaultFileImportOptions( 'quiet' )
quiet_file_import_options = self._new_options.GetDefaultFileImportOptions( FileImportOptions.IMPORT_TYPE_QUIET )
self._quiet_fios = ClientGUIImport.FileImportOptionsButton( default_fios, quiet_file_import_options, show_downloader_options )
self._quiet_fios = ClientGUIImport.FileImportOptionsButton( default_fios, quiet_file_import_options, show_downloader_options, allow_default_selection = False )
loud_file_import_options = self._new_options.GetDefaultFileImportOptions( 'loud' )
loud_file_import_options = self._new_options.GetDefaultFileImportOptions( FileImportOptions.IMPORT_TYPE_LOUD )
self._loud_fios = ClientGUIImport.FileImportOptionsButton( default_fios, loud_file_import_options, show_downloader_options )
self._loud_fios = ClientGUIImport.FileImportOptionsButton( default_fios, loud_file_import_options, show_downloader_options, allow_default_selection = False )
#
rows = []
rows.append( ( 'For \'quiet\' import contexts like import folders and subscriptions:', self._quiet_fios ) )
rows.append( ( 'For import contexts that work on pages:', self._loud_fios ) )
rows.append( ( 'For \'quiet\' import contexts: import folders, subscriptions, Client API:', self._quiet_fios ) )
rows.append( ( 'For \'loud\' import contexts: downloader pages:', self._loud_fios ) )
gridbox = ClientGUICommon.WrapInGrid( default_fios, rows )
@ -1712,8 +1713,8 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
def UpdateOptions( self ):
self._new_options.SetDefaultFileImportOptions( 'quiet', self._quiet_fios.GetValue() )
self._new_options.SetDefaultFileImportOptions( 'loud', self._loud_fios.GetValue() )
self._new_options.SetDefaultFileImportOptions( FileImportOptions.IMPORT_TYPE_QUIET, self._quiet_fios.GetValue() )
self._new_options.SetDefaultFileImportOptions( FileImportOptions.IMPORT_TYPE_LOUD, self._loud_fios.GetValue() )

5
hydrus/client/gui/ClientGUIScrolledPanelsReview.py
View File

@ -53,6 +53,7 @@ from hydrus.client.gui.lists import ClientGUIListCtrl
from hydrus.client.gui.search import ClientGUIACDropdown
from hydrus.client.gui.widgets import ClientGUICommon
from hydrus.client.gui.widgets import ClientGUIMenuButton
from hydrus.client.importing.options import FileImportOptions
from hydrus.client.metadata import ClientTags
from hydrus.client.networking import ClientNetworkingContexts
from hydrus.client.networking import ClientNetworkingDomain
@ -3106,7 +3107,9 @@ class ReviewLocalFileImports( ClientGUIScrolledPanels.ReviewPanel ):
self._progress_cancel = ClientGUICommon.BetterBitmapButton( self, CC.global_pixmaps().stop, self.StopProgress )
self._progress_cancel.setEnabled( False )
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
show_downloader_options = False
self._file_import_options = ClientGUIImport.FileImportOptionsButton( self, file_import_options, show_downloader_options )

37
hydrus/client/gui/ClientGUISubscriptions.py
View File

@ -1353,6 +1353,7 @@ class EditSubscriptionsPanel( ClientGUIScrolledPanels.EditPanel ):
self._subscriptions_panel.AddButton( 'select subscriptions', self.SelectSubscriptions )
self._subscriptions_panel.AddButton( 'overwrite checker timings', self.SetCheckerOptions, enabled_only_on_selection = True )
self._subscriptions_panel.AddButton( 'overwrite file import options', self.SetFileImportOptions, enabled_only_on_selection = True )
self._subscriptions_panel.AddButton( 'overwrite tag import options', self.SetTagImportOptions, enabled_only_on_selection = True )
#
@ -2821,6 +2822,39 @@ class EditSubscriptionsPanel( ClientGUIScrolledPanels.EditPanel ):
def SetFileImportOptions( self ):
subscriptions = self._subscriptions.GetData( only_selected = True )
if len( subscriptions ) == 0:
return
file_import_options = subscriptions[0].GetFileImportOptions()
show_downloader_options = True
allow_default_selection = True
with ClientGUITopLevelWindowsPanels.DialogEdit( self, 'edit tag import options' ) as dlg:
panel = ClientGUIScrolledPanelsEdit.EditFileImportOptions( dlg, file_import_options, show_downloader_options, allow_default_selection )
dlg.SetPanel( panel )
if dlg.exec() == QW.QDialog.Accepted:
file_import_options = panel.GetValue()
for subscription in subscriptions:
subscription.SetFileImportOptions( file_import_options )
self._subscriptions.UpdateDatas( subscriptions )
def SetTagImportOptions( self ):
subscriptions = self._subscriptions.GetData( only_selected = True )
@ -2832,10 +2866,11 @@ class EditSubscriptionsPanel( ClientGUIScrolledPanels.EditPanel ):
tag_import_options = subscriptions[0].GetTagImportOptions()
show_downloader_options = True
allow_default_selection = True
with ClientGUITopLevelWindowsPanels.DialogEdit( self, 'edit tag import options' ) as dlg:
panel = ClientGUIScrolledPanelsEdit.EditTagImportOptionsPanel( dlg, tag_import_options, show_downloader_options, allow_default_selection = True )
panel = ClientGUIScrolledPanelsEdit.EditTagImportOptionsPanel( dlg, tag_import_options, show_downloader_options, allow_default_selection )
dlg.SetPanel( panel )

15
hydrus/client/gui/QtPorting.py
View File

@ -1112,7 +1112,7 @@ def AdjustOpacity( image, opacity_factor ):
def ToKeySequence( modifiers, key ):
if qtpy.PYQT5 or qtpy.PYSIDE2:
if QtInit.WE_ARE_QT5:
if isinstance( modifiers, QC.Qt.KeyboardModifiers ):
@ -1122,16 +1122,21 @@ def ToKeySequence( modifiers, key ):
if modifiers & modifier: seq_str += QG.QKeySequence( modifier ).toString()
seq_str += QG.QKeySequence( key ).toString()
return QG.QKeySequence( seq_str )
else: return QG.QKeySequence( key + modifiers )
else:
return QG.QKeySequence( key + modifiers )
else:
return QG.QKeySequence( QC.QKeyCombination( modifiers, key ) )
def AddShortcut( widget, modifier, key, callable, *args ):

10
hydrus/client/gui/pages/ClientGUIManagement.py
View File

@ -2665,6 +2665,16 @@ class ManagementPanelImporterMultipleGallery( ManagementPanelImporter ):
self._multiple_gallery_import.SetFileLimit( self._file_limit.GetValue() )
def PendSubscriptionGapDownloader( self, gug_key_and_name, query_text, file_import_options, tag_import_options, file_limit ):
new_query = self._multiple_gallery_import.PendSubscriptionGapDownloader( gug_key_and_name, query_text, file_import_options, tag_import_options, file_limit )
if new_query is not None and self._highlighted_gallery_import is None and HG.client_controller.new_options.GetBoolean( 'highlight_new_query' ):
self._HighlightGalleryImport( new_query )
def SetSearchFocus( self ):
ClientGUIFunctions.SetFocusLater( self._query_input )

15
hydrus/client/gui/pages/ClientGUIResults.py
View File

@ -2936,11 +2936,22 @@ class MediaPanelThumbnails( MediaPanel ):
column_index = x // t_span_x
row_index = y // t_span_y
if column_index >= self._num_columns: return None
if column_index >= self._num_columns:
return None
thumbnail_index = self._num_columns * row_index + column_index
if thumbnail_index >= len( self._sorted_media ): return None
if thumbnail_index < 0:
return None
if thumbnail_index >= len( self._sorted_media ):
return None
return self._sorted_media[ thumbnail_index ]

3
hydrus/client/gui/search/ClientGUIACDropdown.py
View File

@ -2215,6 +2215,9 @@ class AutoCompleteDropdownTagsRead( AutoCompleteDropdownTags ):
self._SetLocationContext( self._file_search_context.GetLocationContext() )
self._SetTagService( self._file_search_context.GetTagContext().service_key )
self._include_current_tags.SetOnOff( self._file_search_context.GetTagContext().include_current_tags )
self._include_pending_tags.SetOnOff( self._file_search_context.GetTagContext().include_pending_tags )
self._SignalNewSearchState()

18
hydrus/client/gui/search/ClientGUIPredicatesSingle.py
View File

@ -1440,14 +1440,16 @@ class PanelPredicateSystemMime( PanelPredicateSystemSingle ):
predicate = self._GetPredicateToInitialisePanelWith( predicate )
mimes = predicate.GetValue()
summary_mimes = predicate.GetValue()
if isinstance( mimes, int ):
if isinstance( summary_mimes, int ):
mimes = ( mimes, )
summary_mimes = ( summary_mimes, )
self._mimes.SetValue( mimes )
specific_mimes = ClientSearch.ConvertSummaryFiletypesToSpecific( summary_mimes )
self._mimes.SetValue( specific_mimes )
#
@ -1463,16 +1465,16 @@ class PanelPredicateSystemMime( PanelPredicateSystemSingle ):
def GetDefaultPredicate( self ):
mimes = tuple()
specific_mimes = tuple()
return ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_MIME, mimes )
return ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_MIME, specific_mimes )
def GetPredicates( self ):
mimes = self._mimes.GetValue()
specific_mimes = self._mimes.GetValue()
predicates = ( ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_MIME, mimes ), )
predicates = ( ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_MIME, specific_mimes ), )
return predicates

46
hydrus/client/importing/ClientImportFileSeeds.py
View File

@ -224,6 +224,20 @@ class FileSeed( HydrusSerialisable.SerialisableBase ):
self._hashes = { hash_type : bytes.fromhex( encoded_hash ) for ( hash_type, encoded_hash ) in serialisable_hashes if encoded_hash is not None }
def _SetupFileImportOptions( self, given_file_import_options: FileImportOptions.FileImportOptions, loud_or_quiet: int ) -> FileImportOptions.FileImportOptions:
if given_file_import_options.IsDefault():
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( loud_or_quiet )
else:
file_import_options = given_file_import_options
return file_import_options
def _SetupTagImportOptions( self, given_tag_import_options: TagImportOptions.TagImportOptions ) -> TagImportOptions.TagImportOptions:
if given_tag_import_options.IsDefault():
@ -394,7 +408,9 @@ class FileSeed( HydrusSerialisable.SerialisableBase ):
self._CheckTagsVeto( self._tags, tag_import_options )
def DownloadAndImportRawFile( self, file_url: str, file_import_options, network_job_factory, network_job_presentation_context_factory, status_hook, override_bandwidth = False, forced_referral_url = None, file_seed_cache = None ):
def DownloadAndImportRawFile( self, file_url: str, file_import_options, loud_or_quiet: int, network_job_factory, network_job_presentation_context_factory, status_hook, override_bandwidth = False, forced_referral_url = None, file_seed_cache = None ):
file_import_options = self._SetupFileImportOptions( file_import_options, loud_or_quiet )
self.AddPrimaryURLs( ( file_url, ) )
@ -808,10 +824,12 @@ class FileSeed( HydrusSerialisable.SerialisableBase ):
self.SetHash( file_import_status.hash )
def ImportPath( self, file_seed_cache: "FileSeedCache", file_import_options: FileImportOptions.FileImportOptions, limited_mimes = None, status_hook = None ):
def ImportPath( self, file_seed_cache: "FileSeedCache", file_import_options: FileImportOptions.FileImportOptions, loud_or_quiet: int, status_hook = None ):
try:
file_import_options = self._SetupFileImportOptions( file_import_options, loud_or_quiet )
if self.file_seed_type != FILE_SEED_TYPE_HDD:
raise HydrusExceptions.VetoException( 'Attempted to import as a path, but I do not think I am a path!' )
@ -840,23 +858,6 @@ class FileSeed( HydrusSerialisable.SerialisableBase ):
raise Exception( 'File failed to copy to temp path--see log for error.' )
if limited_mimes is not None:
# I think this thing should and will be rolled into file import options late
if status_hook is not None:
status_hook( 'testing file type' )
mime = HydrusFileHandling.GetMime( temp_path )
if mime not in limited_mimes:
raise HydrusExceptions.VetoException( 'Not in allowed mimes!' )
self.Import( temp_path, file_import_options, status_hook = status_hook )
finally:
@ -1131,7 +1132,7 @@ class FileSeed( HydrusSerialisable.SerialisableBase ):
return False
def WorkOnURL( self, file_seed_cache: "FileSeedCache", status_hook, network_job_factory, network_job_presentation_context_factory, file_import_options: FileImportOptions.FileImportOptions, tag_import_options: TagImportOptions.TagImportOptions ):
def WorkOnURL( self, file_seed_cache: "FileSeedCache", status_hook, network_job_factory, network_job_presentation_context_factory, file_import_options: FileImportOptions.FileImportOptions, loud_or_quiet: int, tag_import_options: TagImportOptions.TagImportOptions ):
did_substantial_work = False
@ -1149,6 +1150,7 @@ class FileSeed( HydrusSerialisable.SerialisableBase ):
raise HydrusExceptions.VetoException( 'Cannot parse {}: {}'.format( match_name, cannot_parse_reason ) )
file_import_options = self._SetupFileImportOptions( file_import_options, loud_or_quiet )
tag_import_options = self._SetupTagImportOptions( tag_import_options )
status_hook( 'checking url status' )
@ -1326,7 +1328,7 @@ class FileSeed( HydrusSerialisable.SerialisableBase ):
if should_download_file:
self.DownloadAndImportRawFile( file_url, file_import_options, network_job_factory, network_job_presentation_context_factory, status_hook, override_bandwidth = True, forced_referral_url = url_for_child_referral, file_seed_cache = file_seed_cache )
self.DownloadAndImportRawFile( file_url, file_import_options, loud_or_quiet, network_job_factory, network_job_presentation_context_factory, status_hook, override_bandwidth = True, forced_referral_url = url_for_child_referral, file_seed_cache = file_seed_cache )
elif url_type == HC.URL_TYPE_POST and can_parse:
@ -1403,7 +1405,7 @@ class FileSeed( HydrusSerialisable.SerialisableBase ):
file_url = self.file_seed_data
self.DownloadAndImportRawFile( file_url, file_import_options, network_job_factory, network_job_presentation_context_factory, status_hook, file_seed_cache = file_seed_cache )
self.DownloadAndImportRawFile( file_url, file_import_options, loud_or_quiet, network_job_factory, network_job_presentation_context_factory, status_hook, file_seed_cache = file_seed_cache )

57
hydrus/client/importing/ClientImportGallery.py
View File

@ -1,5 +1,6 @@
import threading
import time
import typing
from hydrus.core import HydrusConstants as HC
from hydrus.core import HydrusData
@ -63,7 +64,8 @@ class GalleryImport( HydrusSerialisable.SerialisableBase ):
self._files_paused = start_file_queue_paused
self._gallery_paused = start_gallery_queue_paused
self._file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
self._file_import_options = FileImportOptions.FileImportOptions()
self._file_import_options.SetIsDefault( True )
self._tag_import_options = TagImportOptions.TagImportOptions( is_default = True )
@ -242,7 +244,7 @@ class GalleryImport( HydrusSerialisable.SerialisableBase ):
did_substantial_work = file_seed.WorkOnURL( self._file_seed_cache, status_hook, self._NetworkJobFactory, self._FileNetworkJobPresentationContextFactory, self._file_import_options, self._tag_import_options )
did_substantial_work = file_seed.WorkOnURL( self._file_seed_cache, status_hook, self._NetworkJobFactory, self._FileNetworkJobPresentationContextFactory, self._file_import_options, FileImportOptions.IMPORT_TYPE_LOUD, self._tag_import_options )
with self._lock:
@ -590,13 +592,15 @@ class GalleryImport( HydrusSerialisable.SerialisableBase ):
gallery_go = gallery_work_to_do and not self._gallery_paused
files_go = files_work_to_do and not self._files_paused
work_is_going_on = self._gallery_working_lock.locked() or self._files_working_lock.locked()
if not ( gallery_work_to_do or files_work_to_do ):
return ( ClientImporting.DOWNLOADER_SIMPLE_STATUS_DONE, 'DONE' )
elif gallery_go or files_go:
if self._gallery_working_lock.locked() or self._files_working_lock.locked():
if work_is_going_on:
return ( ClientImporting.DOWNLOADER_SIMPLE_STATUS_WORKING, 'working' )
@ -607,7 +611,14 @@ class GalleryImport( HydrusSerialisable.SerialisableBase ):
else:
return ( ClientImporting.DOWNLOADER_SIMPLE_STATUS_PAUSED, '' )
if work_is_going_on:
return ( ClientImporting.DOWNLOADER_SIMPLE_STATUS_PAUSING, 'pausing\u2026' )
else:
return ( ClientImporting.DOWNLOADER_SIMPLE_STATUS_PAUSED, '' )
@ -1026,7 +1037,9 @@ class MultipleGalleryImport( HydrusSerialisable.SerialisableBase ):
self._start_gallery_queues_paused = False
self._merge_simultaneous_pends_to_one_importer = False
self._file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
self._file_import_options = FileImportOptions.FileImportOptions()
self._file_import_options.SetIsDefault( True )
self._tag_import_options = TagImportOptions.TagImportOptions( is_default = True )
self._gallery_imports = HydrusSerialisable.SerialisableList()
@ -1061,11 +1074,6 @@ class MultipleGalleryImport( HydrusSerialisable.SerialisableBase ):
self._gallery_import_keys_to_gallery_imports[ gallery_import_key ] = gallery_import
if len( self._gallery_imports ) == 1:
self._highlighted_gallery_import_key = gallery_import_key
def _GetSerialisableInfo( self ):
@ -1152,6 +1160,18 @@ class MultipleGalleryImport( HydrusSerialisable.SerialisableBase ):
self._last_serialisable_change_timestamp = HydrusData.GetNow()
def _SetHighlightedGalleryImport( self, highlighted_gallery_import: GalleryImport ):
highlighted_gallery_import_key = highlighted_gallery_import.GetGalleryImportKey()
if highlighted_gallery_import_key != self._highlighted_gallery_import_key:
self._highlighted_gallery_import_key = highlighted_gallery_import.GetGalleryImportKey()
self._SerialisableChangeMade()
def _SetStatusDirty( self ):
self._status_dirty = True
@ -1460,7 +1480,7 @@ class MultipleGalleryImport( HydrusSerialisable.SerialisableBase ):
def PendSubscriptionGapDownloader( self, gug_key_and_name, query_text, file_import_options, tag_import_options, file_limit ):
def PendSubscriptionGapDownloader( self, gug_key_and_name, query_text, file_import_options, tag_import_options, file_limit ) -> typing.Optional[ GalleryImport ]:
with self._lock:
@ -1470,7 +1490,7 @@ class MultipleGalleryImport( HydrusSerialisable.SerialisableBase ):
HydrusData.ShowText( 'Could not find a Gallery URL Generator for "{}"!'.format( self._gug_key_and_name[1] ) )
return
return None
initial_search_urls = gug.GenerateGalleryURLs( query_text )
@ -1479,7 +1499,7 @@ class MultipleGalleryImport( HydrusSerialisable.SerialisableBase ):
HydrusData.ShowText( 'The Gallery URL Generator "{}" did not produce any URLs!'.format( self._gug_key_and_name[1] ) )
return
return None
gallery_import = GalleryImport( query = query_text, source_name = gug_key_and_name[1], initial_search_urls = initial_search_urls, start_file_queue_paused = self._start_file_queues_paused, start_gallery_queue_paused = self._start_gallery_queues_paused )
@ -1504,6 +1524,8 @@ class MultipleGalleryImport( HydrusSerialisable.SerialisableBase ):
self._SerialisableChangeMade()
return gallery_import
def PendQueries( self, query_texts ):
@ -1644,14 +1666,7 @@ class MultipleGalleryImport( HydrusSerialisable.SerialisableBase ):
with self._lock:
highlighted_gallery_import_key = highlighted_gallery_import.GetGalleryImportKey()
if highlighted_gallery_import_key != self._highlighted_gallery_import_key:
self._highlighted_gallery_import_key = highlighted_gallery_import.GetGalleryImportKey()
self._SerialisableChangeMade()
self._SetHighlightedGalleryImport( highlighted_gallery_import )

48
hydrus/client/importing/ClientImportLocal.py
View File

@ -15,6 +15,7 @@ from hydrus.client import ClientConstants as CC
from hydrus.client import ClientData
from hydrus.client import ClientFiles
from hydrus.client import ClientPaths
from hydrus.client import ClientSearch
from hydrus.client import ClientThreading
from hydrus.client.importing import ClientImportControl
from hydrus.client.importing import ClientImporting
@ -159,7 +160,7 @@ class HDDImport( HydrusSerialisable.SerialisableBase ):
file_seed.ImportPath( self._file_seed_cache, self._file_import_options, status_hook = status_hook )
file_seed.ImportPath( self._file_seed_cache, self._file_import_options, FileImportOptions.IMPORT_TYPE_LOUD, status_hook = status_hook )
if file_seed.status in CC.SUCCESSFUL_IMPORT_STATES:
@ -387,18 +388,14 @@ class ImportFolder( HydrusSerialisable.SerialisableBaseNamed ):
SERIALISABLE_TYPE = HydrusSerialisable.SERIALISABLE_TYPE_IMPORT_FOLDER
SERIALISABLE_NAME = 'Import Folder'
SERIALISABLE_VERSION = 6
SERIALISABLE_VERSION = 7
def __init__( self, name, path = '', file_import_options = None, tag_import_options = None, tag_service_keys_to_filename_tagging_options = None, mimes = None, actions = None, action_locations = None, period = 3600, check_regularly = True, show_working_popup = True, publish_files_to_popup_button = True, publish_files_to_page = False ):
if mimes is None:
mimes = HC.ALLOWED_MIMES
def __init__( self, name, path = '', file_import_options = None, tag_import_options = None, tag_service_keys_to_filename_tagging_options = None, actions = None, action_locations = None, period = 3600, check_regularly = True, show_working_popup = True, publish_files_to_popup_button = True, publish_files_to_page = False ):
if file_import_options is None:
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'quiet' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
if tag_import_options is None:
@ -429,7 +426,6 @@ class ImportFolder( HydrusSerialisable.SerialisableBaseNamed ):
HydrusSerialisable.SerialisableBaseNamed.__init__( self, name )
self._path = path
self._mimes = mimes
self._file_import_options = file_import_options
self._tag_import_options = tag_import_options
self._tag_service_keys_to_filename_tagging_options = tag_service_keys_to_filename_tagging_options
@ -622,7 +618,7 @@ class ImportFolder( HydrusSerialisable.SerialisableBaseNamed ):
action_pairs = list(self._actions.items())
action_location_pairs = list(self._action_locations.items())
return ( self._path, list( self._mimes ), serialisable_file_import_options, serialisable_tag_import_options, serialisable_tag_service_keys_to_filename_tagging_options, action_pairs, action_location_pairs, self._period, self._check_regularly, serialisable_file_seed_cache, self._last_checked, self._paused, self._check_now, self._show_working_popup, self._publish_files_to_popup_button, self._publish_files_to_page )
return ( self._path, serialisable_file_import_options, serialisable_tag_import_options, serialisable_tag_service_keys_to_filename_tagging_options, action_pairs, action_location_pairs, self._period, self._check_regularly, serialisable_file_seed_cache, self._last_checked, self._paused, self._check_now, self._show_working_popup, self._publish_files_to_popup_button, self._publish_files_to_page )
def _ImportFiles( self, job_key ):
@ -670,7 +666,7 @@ class ImportFolder( HydrusSerialisable.SerialisableBaseNamed ):
path = file_seed.file_seed_data
file_seed.ImportPath( self._file_seed_cache, self._file_import_options, limited_mimes = self._mimes )
file_seed.ImportPath( self._file_seed_cache, self._file_import_options, FileImportOptions.IMPORT_TYPE_QUIET )
if file_seed.status in CC.SUCCESSFUL_IMPORT_STATES:
@ -770,9 +766,7 @@ class ImportFolder( HydrusSerialisable.SerialisableBaseNamed ):
def _InitialiseFromSerialisableInfo( self, serialisable_info ):
( self._path, mimes, serialisable_file_import_options, serialisable_tag_import_options, serialisable_tag_service_keys_to_filename_tagging_options, action_pairs, action_location_pairs, self._period, self._check_regularly, serialisable_file_seed_cache, self._last_checked, self._paused, self._check_now, self._show_working_popup, self._publish_files_to_popup_button, self._publish_files_to_page ) = serialisable_info
self._mimes = set( mimes )
( self._path, serialisable_file_import_options, serialisable_tag_import_options, serialisable_tag_service_keys_to_filename_tagging_options, action_pairs, action_location_pairs, self._period, self._check_regularly, serialisable_file_seed_cache, self._last_checked, self._paused, self._check_now, self._show_working_popup, self._publish_files_to_popup_button, self._publish_files_to_page ) = serialisable_info
self._actions = dict( action_pairs )
self._action_locations = dict( action_location_pairs )
@ -860,6 +854,21 @@ class ImportFolder( HydrusSerialisable.SerialisableBaseNamed ):
return ( 6, new_serialisable_info )
if version == 6:
( path, mimes, serialisable_file_import_options, serialisable_tag_import_options, serialisable_tag_service_keys_to_filename_tagging_options, action_pairs, action_location_pairs, period, check_regularly, serialisable_file_seed_cache, last_checked, paused, check_now, show_working_popup, publish_files_to_popup_button, publish_files_to_page ) = old_serialisable_info
file_import_options = HydrusSerialisable.CreateFromSerialisableTuple( serialisable_file_import_options )
file_import_options.SetAllowedSpecificFiletypes( mimes )
serialisable_file_import_options = file_import_options.GetSerialisableTuple()
new_serialisable_info = ( path, serialisable_file_import_options, serialisable_tag_import_options, serialisable_tag_service_keys_to_filename_tagging_options, action_pairs, action_location_pairs, period, check_regularly, serialisable_file_seed_cache, last_checked, paused, check_now, show_working_popup, publish_files_to_popup_button, publish_files_to_page )
return ( 7, new_serialisable_info )
def CheckNow( self ):
@ -972,7 +981,7 @@ class ImportFolder( HydrusSerialisable.SerialisableBaseNamed ):
def ToTuple( self ):
return ( self._name, self._path, self._mimes, self._file_import_options, self._tag_import_options, self._tag_service_keys_to_filename_tagging_options, self._actions, self._action_locations, self._period, self._check_regularly, self._paused, self._check_now, self._show_working_popup, self._publish_files_to_popup_button, self._publish_files_to_page )
return ( self._name, self._path, self._file_import_options, self._tag_import_options, self._tag_service_keys_to_filename_tagging_options, self._actions, self._action_locations, self._period, self._check_regularly, self._paused, self._check_now, self._show_working_popup, self._publish_files_to_popup_button, self._publish_files_to_page )
def SetFileSeedCache( self, file_seed_cache ):
@ -980,23 +989,22 @@ class ImportFolder( HydrusSerialisable.SerialisableBaseNamed ):
self._file_seed_cache = file_seed_cache
def SetTuple( self, name, path, mimes, file_import_options, tag_import_options, tag_service_keys_to_filename_tagging_options, actions, action_locations, period, check_regularly, paused, check_now, show_working_popup, publish_files_to_popup_button, publish_files_to_page ):
def SetTuple( self, name, path, file_import_options, tag_import_options, tag_service_keys_to_filename_tagging_options, actions, action_locations, period, check_regularly, paused, check_now, show_working_popup, publish_files_to_popup_button, publish_files_to_page ):
if path != self._path:
self._file_seed_cache = ClientImportFileSeeds.FileSeedCache()
mimes = set( mimes )
mimes = set( file_import_options.GetAllowedSpecificFiletypes() )
if mimes != self._mimes:
if mimes != set( self._file_import_options.GetAllowedSpecificFiletypes() ):
self._file_seed_cache.RemoveFileSeedsByStatus( ( CC.STATUS_VETOED, ) )
self._name = name
self._path = path
self._mimes = mimes
self._file_import_options = file_import_options
self._tag_import_options = tag_import_options
self._tag_service_keys_to_filename_tagging_options = tag_service_keys_to_filename_tagging_options

16
hydrus/client/importing/ClientImportSimpleURLs.py
View File

@ -28,12 +28,13 @@ class SimpleDownloaderImport( HydrusSerialisable.SerialisableBase ):
HydrusSerialisable.SerialisableBase.__init__( self )
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
self._pending_jobs = []
self._gallery_seed_log = ClientImportGallerySeeds.GallerySeedLog()
self._file_seed_cache = ClientImportFileSeeds.FileSeedCache()
self._file_import_options = file_import_options
self._file_import_options = FileImportOptions.FileImportOptions()
self._file_import_options.SetIsDefault( True )
self._formula_name = 'all files linked by images in page'
self._gallery_paused = False
self._files_paused = False
@ -224,7 +225,7 @@ class SimpleDownloaderImport( HydrusSerialisable.SerialisableBase ):
try:
did_substantial_work = file_seed.WorkOnURL( self._file_seed_cache, status_hook, self._NetworkJobFactory, self._FileNetworkJobPresentationContextFactory, self._file_import_options, tag_import_options )
did_substantial_work = file_seed.WorkOnURL( self._file_seed_cache, status_hook, self._NetworkJobFactory, self._FileNetworkJobPresentationContextFactory, self._file_import_options, FileImportOptions.IMPORT_TYPE_LOUD, tag_import_options )
except HydrusExceptions.NetworkException as e:
@ -829,7 +830,10 @@ class URLsImport( HydrusSerialisable.SerialisableBase ):
self._gallery_seed_log = ClientImportGallerySeeds.GallerySeedLog()
self._file_seed_cache = ClientImportFileSeeds.FileSeedCache()
self._file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
self._file_import_options = FileImportOptions.FileImportOptions()
self._file_import_options.SetIsDefault( True )
self._tag_import_options = TagImportOptions.TagImportOptions( is_default = True )
self._paused = False
@ -989,7 +993,7 @@ class URLsImport( HydrusSerialisable.SerialisableBase ):
status_hook = lambda s: s # do nothing for now
did_substantial_work = file_seed.WorkOnURL( self._file_seed_cache, status_hook, self._NetworkJobFactory, self._FileNetworkJobPresentationContextFactory, self._file_import_options, self._tag_import_options )
did_substantial_work = file_seed.WorkOnURL( self._file_seed_cache, status_hook, self._NetworkJobFactory, self._FileNetworkJobPresentationContextFactory, self._file_import_options, FileImportOptions.IMPORT_TYPE_LOUD, self._tag_import_options )
if file_seed.ShouldPresent( self._file_import_options.GetPresentationImportOptions() ):

10
hydrus/client/importing/ClientImportSubscriptionLegacy.py
View File

@ -504,7 +504,9 @@ class SubscriptionLegacy( HydrusSerialisable.SerialisableBaseNamed ):
self._periodic_file_limit = 100
self._paused = False
self._file_import_options = new_options.GetDefaultFileImportOptions( 'quiet' )
self._file_import_options = FileImportOptions.FileImportOptions()
self._file_import_options.SetIsDefault( True )
self._tag_import_options = TagImportOptions.TagImportOptions( is_default = True )
self._no_work_until = 0
@ -673,7 +675,7 @@ class SubscriptionLegacy( HydrusSerialisable.SerialisableBaseNamed ):
message += os.linesep * 2
message += login_fail_reason
message += os.linesep * 2
message += 'The subscription has paused. Please see if you can fix the problem and then unpause. Hydrus dev would like feedback on this process.'
message += 'The subscription has paused. Please see if you can fix the problem and then unpause. If the login script stopped because of missing cookies or similar, it may be broken. Please check out Hydrus Companion for a better login solution.'
HydrusData.ShowText( message )
@ -733,7 +735,7 @@ class SubscriptionLegacy( HydrusSerialisable.SerialisableBaseNamed ):
message += os.linesep * 2
message += login_fail_reason
message += os.linesep * 2
message += 'The subscription has paused. Please see if you can fix the problem and then unpause. Hydrus dev would like feedback on this process.'
message += 'The subscription has paused. Please see if you can fix the problem and then unpause. If the login script stopped because of missing cookies or similar, it may be broken. Please check out Hydrus Companion for a better login solution.'
HydrusData.ShowText( message )
@ -1000,7 +1002,7 @@ class SubscriptionLegacy( HydrusSerialisable.SerialisableBaseNamed ):
job_key.SetVariable( 'popup_text_2', x_out_of_y + text )
file_seed.WorkOnURL( file_seed_cache, status_hook, self._GenerateNetworkJobFactory( query ), ClientImporting.GenerateMultiplePopupNetworkJobPresentationContextFactory( job_key ), self._file_import_options, self._tag_import_options )
file_seed.WorkOnURL( file_seed_cache, status_hook, self._GenerateNetworkJobFactory( query ), ClientImporting.GenerateMultiplePopupNetworkJobPresentationContextFactory( job_key ), self._file_import_options, FileImportOptions.IMPORT_TYPE_QUIET, self._tag_import_options )
query_tag_import_options = query.GetTagImportOptions()

20
hydrus/client/importing/ClientImportSubscriptions.py
View File

@ -61,7 +61,9 @@ class Subscription( HydrusSerialisable.SerialisableBaseNamed ):
self._paused = False
self._file_import_options = new_options.GetDefaultFileImportOptions( 'quiet' )
self._file_import_options = FileImportOptions.FileImportOptions()
self._file_import_options.SetIsDefault( True )
self._tag_import_options = TagImportOptions.TagImportOptions( is_default = True )
self._no_work_until = 0
@ -412,7 +414,7 @@ class Subscription( HydrusSerialisable.SerialisableBaseNamed ):
message += os.linesep * 2
message += login_reason
message += os.linesep * 2
message += 'The subscription has paused. Please see if you can fix the problem and then unpause. Hydrus dev would like feedback on this process.'
message += 'The subscription has paused. Please see if you can fix the problem and then unpause. If the login script stopped because of missing cookies or similar, it may be broken. Please check out Hydrus Companion for a better login solution.'
HydrusData.ShowText( message )
@ -973,7 +975,7 @@ class Subscription( HydrusSerialisable.SerialisableBaseNamed ):
message += os.linesep * 2
message += login_reason
message += os.linesep * 2
message += 'The subscription has paused. Please see if you can fix the problem and then unpause. Hydrus dev would like feedback on this process.'
message += 'The subscription has paused. Please see if you can fix the problem and then unpause. If the login script stopped because of missing cookies or similar, it may be broken. Please check out Hydrus Companion for a better login solution.'
HydrusData.ShowText( message )
@ -1011,7 +1013,7 @@ class Subscription( HydrusSerialisable.SerialisableBaseNamed ):
job_key.SetVariable( 'popup_text_2', x_out_of_y + text )
file_seed.WorkOnURL( file_seed_cache, status_hook, query_header.GenerateNetworkJobFactory( self._name ), ClientImporting.GenerateMultiplePopupNetworkJobPresentationContextFactory( job_key ), self._file_import_options, self._tag_import_options )
file_seed.WorkOnURL( file_seed_cache, status_hook, query_header.GenerateNetworkJobFactory( self._name ), ClientImporting.GenerateMultiplePopupNetworkJobPresentationContextFactory( job_key ), self._file_import_options, FileImportOptions.IMPORT_TYPE_QUIET, self._tag_import_options )
query_tag_import_options = query_header.GetTagImportOptions()
@ -1272,6 +1274,11 @@ class Subscription( HydrusSerialisable.SerialisableBaseNamed ):
return self._checker_options
def GetFileImportOptions( self ):
return self._file_import_options
def GetGUGKeyAndName( self ):
return self._gug_key_and_name
@ -1493,6 +1500,11 @@ class Subscription( HydrusSerialisable.SerialisableBaseNamed ):
def SetFileImportOptions( self, file_import_options ):
self._file_import_options = file_import_options.Duplicate()
def SetPresentationOptions( self, show_a_popup_while_working, publish_files_to_popup_button, publish_files_to_page, publish_label_override, merge_query_publish_events ):
self._show_a_popup_while_working = show_a_popup_while_working

30
hydrus/client/importing/ClientImportWatchers.py
View File

@ -39,7 +39,10 @@ class MultipleWatcherImport( HydrusSerialisable.SerialisableBase ):
self._highlighted_watcher_url = None
self._checker_options = HG.client_controller.new_options.GetDefaultWatcherCheckerOptions()
self._file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
self._file_import_options = FileImportOptions.FileImportOptions()
self._file_import_options.SetIsDefault( True )
self._tag_import_options = TagImportOptions.TagImportOptions( is_default = True )
self._watcher_keys_to_watchers = {}
@ -182,7 +185,10 @@ class MultipleWatcherImport( HydrusSerialisable.SerialisableBase ):
try:
checker_options = HG.client_controller.new_options.GetDefaultWatcherCheckerOptions()
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
tag_import_options = TagImportOptions.TagImportOptions( is_default = True )
except:
@ -629,7 +635,10 @@ class WatcherImport( HydrusSerialisable.SerialisableBase ):
self._external_additional_service_keys_to_tags = ClientTags.ServiceKeysToTags()
self._checker_options = HG.client_controller.new_options.GetDefaultWatcherCheckerOptions()
self._file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
self._file_import_options = FileImportOptions.FileImportOptions()
self._file_import_options.SetIsDefault( True )
self._tag_import_options = TagImportOptions.TagImportOptions( is_default = True )
self._last_check_time = 0
self._checking_status = ClientImporting.CHECKER_STATUS_OK
@ -1070,7 +1079,7 @@ class WatcherImport( HydrusSerialisable.SerialisableBase ):
did_substantial_work = file_seed.WorkOnURL( self._file_seed_cache, status_hook, self._NetworkJobFactory, self._FileNetworkJobPresentationContextFactory, self._file_import_options, self._tag_import_options )
did_substantial_work = file_seed.WorkOnURL( self._file_seed_cache, status_hook, self._NetworkJobFactory, self._FileNetworkJobPresentationContextFactory, self._file_import_options, FileImportOptions.IMPORT_TYPE_LOUD, self._tag_import_options )
with self._lock:
@ -1304,6 +1313,8 @@ class WatcherImport( HydrusSerialisable.SerialisableBase ):
checker_go = HydrusData.TimeHasPassed( self._next_check_time ) and not self._checking_paused
files_go = files_work_to_do and not self._files_paused
work_is_going_on = self._checker_working_lock.locked() or self._files_working_lock.locked()
if not HydrusData.TimeHasPassed( self._no_work_until ):
if self._next_check_time is None:
@ -1319,7 +1330,7 @@ class WatcherImport( HydrusSerialisable.SerialisableBase ):
elif checker_go or files_go:
if self._checker_working_lock.locked() or self._files_working_lock.locked():
if work_is_going_on:
return ( ClientImporting.DOWNLOADER_SIMPLE_STATUS_WORKING, 'working' )
@ -1340,7 +1351,14 @@ class WatcherImport( HydrusSerialisable.SerialisableBase ):
if self._checking_paused:
return ( ClientImporting.DOWNLOADER_SIMPLE_STATUS_PAUSED, '' )
if work_is_going_on:
return ( ClientImporting.DOWNLOADER_SIMPLE_STATUS_PAUSING, 'pausing\u2026' )
else:
return ( ClientImporting.DOWNLOADER_SIMPLE_STATUS_PAUSED, '' )
else:

15
hydrus/client/importing/ClientImporting.py
View File

@ -8,6 +8,7 @@ from hydrus.client import ClientConstants as CC
from hydrus.client import ClientParsing
from hydrus.client import ClientThreading
from hydrus.client.importing import ClientImportFileSeeds
from hydrus.client.importing.options import FileImportOptions
from hydrus.client.networking import ClientNetworkingJobs
CHECKER_STATUS_OK = 0
@ -19,13 +20,15 @@ DOWNLOADER_SIMPLE_STATUS_WORKING = 1
DOWNLOADER_SIMPLE_STATUS_PENDING = 2
DOWNLOADER_SIMPLE_STATUS_PAUSED = 3
DOWNLOADER_SIMPLE_STATUS_DEFERRED = 4
DOWNLOADER_SIMPLE_STATUS_PAUSING = 5
downloader_enum_sort_lookup = {
DOWNLOADER_SIMPLE_STATUS_DONE : 0,
DOWNLOADER_SIMPLE_STATUS_WORKING : 1,
DOWNLOADER_SIMPLE_STATUS_PENDING : 2,
DOWNLOADER_SIMPLE_STATUS_DEFERRED : 3,
DOWNLOADER_SIMPLE_STATUS_PAUSED : 4
DOWNLOADER_SIMPLE_STATUS_PAUSING : 5,
DOWNLOADER_SIMPLE_STATUS_PAUSED : 6
}
DID_SUBSTANTIAL_FILE_WORK_MINIMUM_SLEEP_TIME = 0.1
@ -130,7 +133,8 @@ def THREADDownloadURL( job_key, url, url_string ):
#
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
def network_job_factory( *args, **kwargs ):
@ -159,7 +163,7 @@ def THREADDownloadURL( job_key, url, url_string ):
try:
file_seed.DownloadAndImportRawFile( url, file_import_options, network_job_factory, network_job_presentation_context_factory, status_hook )
file_seed.DownloadAndImportRawFile( url, file_import_options, FileImportOptions.IMPORT_TYPE_LOUD, network_job_factory, network_job_presentation_context_factory, status_hook )
status = file_seed.status
@ -204,7 +208,8 @@ def THREADDownloadURLs( job_key: ClientThreading.JobKey, urls, title ):
presentation_hashes = []
presentation_hashes_fast = set()
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
def network_job_factory( *args, **kwargs ):
@ -243,7 +248,7 @@ def THREADDownloadURLs( job_key: ClientThreading.JobKey, urls, title ):
try:
file_seed.DownloadAndImportRawFile( url, file_import_options, network_job_factory, network_job_presentation_context_factory, status_hook )
file_seed.DownloadAndImportRawFile( url, file_import_options, FileImportOptions.IMPORT_TYPE_LOUD, network_job_factory, network_job_presentation_context_factory, status_hook )
status = file_seed.status

87
hydrus/client/importing/options/FileImportOptions.py
View File

@ -1,4 +1,5 @@
import os
import typing
from hydrus.core import HydrusConstants as HC
from hydrus.core import HydrusData
@ -7,13 +8,17 @@ from hydrus.core import HydrusSerialisable
from hydrus.client import ClientConstants as CC
from hydrus.client import ClientLocation
from hydrus.client import ClientSearch
from hydrus.client.importing.options import PresentationImportOptions
IMPORT_TYPE_QUIET = 0
IMPORT_TYPE_LOUD = 1
class FileImportOptions( HydrusSerialisable.SerialisableBase ):
SERIALISABLE_TYPE = HydrusSerialisable.SERIALISABLE_TYPE_FILE_IMPORT_OPTIONS
SERIALISABLE_NAME = 'File Import Options'
SERIALISABLE_VERSION = 7
SERIALISABLE_VERSION = 8
def __init__( self ):
@ -23,6 +28,7 @@ class FileImportOptions( HydrusSerialisable.SerialisableBase ):
self._do_not_check_known_urls_before_importing = False
self._do_not_check_hashes_before_importing = False
self._allow_decompression_bombs = True
self._filetype_filter_predicate = ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_MIME, value = set( HC.GENERAL_FILETYPES ) )
self._min_size = None
self._max_size = None
self._max_gif_size = None
@ -34,26 +40,32 @@ class FileImportOptions( HydrusSerialisable.SerialisableBase ):
self._presentation_import_options = PresentationImportOptions.PresentationImportOptions()
self._import_destination_location_context = ClientLocation.LocationContext.STATICCreateSimple( CC.LOCAL_FILE_SERVICE_KEY )
self._is_default = False
def _GetSerialisableInfo( self ):
serialisable_import_destination_location_context = self._import_destination_location_context.GetSerialisableTuple()
pre_import_options = ( self._exclude_deleted, self._do_not_check_known_urls_before_importing, self._do_not_check_hashes_before_importing, self._allow_decompression_bombs, self._min_size, self._max_size, self._max_gif_size, self._min_resolution, self._max_resolution, serialisable_import_destination_location_context )
serialisable_filetype_filter_predicate = self._filetype_filter_predicate.GetSerialisableTuple()
pre_import_options = ( self._exclude_deleted, self._do_not_check_known_urls_before_importing, self._do_not_check_hashes_before_importing, self._allow_decompression_bombs, serialisable_filetype_filter_predicate, self._min_size, self._max_size, self._max_gif_size, self._min_resolution, self._max_resolution, serialisable_import_destination_location_context )
post_import_options = ( self._automatic_archive, self._associate_primary_urls, self._associate_source_urls )
serialisable_presentation_import_options = self._presentation_import_options.GetSerialisableTuple()
return ( pre_import_options, post_import_options, serialisable_presentation_import_options )
return ( pre_import_options, post_import_options, serialisable_presentation_import_options, self._is_default )
def _InitialiseFromSerialisableInfo( self, serialisable_info ):
( pre_import_options, post_import_options, serialisable_presentation_import_options ) = serialisable_info
( pre_import_options, post_import_options, serialisable_presentation_import_options, self._is_default ) = serialisable_info
( self._exclude_deleted, self._do_not_check_known_urls_before_importing, self._do_not_check_hashes_before_importing, self._allow_decompression_bombs, self._min_size, self._max_size, self._max_gif_size, self._min_resolution, self._max_resolution, serialisable_import_destination_location_context ) = pre_import_options
( self._exclude_deleted, self._do_not_check_known_urls_before_importing, self._do_not_check_hashes_before_importing, self._allow_decompression_bombs, serialisable_filetype_filter_predicate, self._min_size, self._max_size, self._max_gif_size, self._min_resolution, self._max_resolution, serialisable_import_destination_location_context ) = pre_import_options
( self._automatic_archive, self._associate_primary_urls, self._associate_source_urls ) = post_import_options
self._presentation_import_options = HydrusSerialisable.CreateFromSerialisableTuple( serialisable_presentation_import_options )
self._filetype_filter_predicate = HydrusSerialisable.CreateFromSerialisableTuple( serialisable_filetype_filter_predicate )
self._import_destination_location_context = HydrusSerialisable.CreateFromSerialisableTuple( serialisable_import_destination_location_context )
@ -180,6 +192,25 @@ class FileImportOptions( HydrusSerialisable.SerialisableBase ):
return ( 7, new_serialisable_info )
if version == 7:
( pre_import_options, post_import_options, serialisable_presentation_import_options ) = old_serialisable_info
( exclude_deleted, do_not_check_known_urls_before_importing, do_not_check_hashes_before_importing, allow_decompression_bombs, min_size, max_size, max_gif_size, min_resolution, max_resolution, serialisable_import_destination_location_context ) = pre_import_options
filetype_filter_predicate = ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_MIME, value = set( HC.GENERAL_FILETYPES ) )
serialisable_filetype_filter_predicate = filetype_filter_predicate.GetSerialisableTuple()
pre_import_options = ( exclude_deleted, do_not_check_known_urls_before_importing, do_not_check_hashes_before_importing, allow_decompression_bombs, serialisable_filetype_filter_predicate, min_size, max_size, max_gif_size, min_resolution, max_resolution, serialisable_import_destination_location_context )
is_default = False
new_serialisable_info = ( pre_import_options, post_import_options, serialisable_presentation_import_options, is_default )
return ( 8, new_serialisable_info )
def AllowsDecompressionBombs( self ):
@ -193,6 +224,13 @@ class FileImportOptions( HydrusSerialisable.SerialisableBase ):
def CheckFileIsValid( self, size, mime, width, height ):
allowed_mimes = self.GetAllowedSpecificFiletypes()
if mime not in allowed_mimes:
raise HydrusExceptions.FileImportRulesException( 'File was a {}, which is not allowed by the File Import Options.'.format( HC.mime_string_lookup[ mime ] ) )
if self._min_size is not None and size < self._min_size:
raise HydrusExceptions.FileImportRulesException( 'File was ' + HydrusData.ToHumanBytes( size ) + ' but the lower limit is ' + HydrusData.ToHumanBytes( self._min_size ) + '.' )
@ -235,6 +273,14 @@ class FileImportOptions( HydrusSerialisable.SerialisableBase ):
def CheckReadyToImport( self ) -> None:
if self._import_destination_location_context.IsEmpty():
raise HydrusExceptions.FileImportBlockException( 'There is no import destination set in the File Import Options!' )
def CheckNetworkDownload( self, possible_mime, num_bytes, is_complete_file_size ):
if is_complete_file_size:
@ -283,6 +329,11 @@ class FileImportOptions( HydrusSerialisable.SerialisableBase ):
return self._exclude_deleted
def GetAllowedSpecificFiletypes( self ) -> typing.Collection[ int ]:
return ClientSearch.ConvertSummaryFiletypesToSpecific( self._filetype_filter_predicate.GetValue(), only_searchable = False )
def GetDestinationLocationContext( self ) -> ClientLocation.LocationContext:
return self._import_destination_location_context
@ -302,8 +353,15 @@ class FileImportOptions( HydrusSerialisable.SerialisableBase ):
def GetSummary( self ):
if self._is_default:
return 'Using whatever the default file import options is at at time of import.'
statements = []
statements.append( 'allowing {}'.format( ClientSearch.ConvertSummaryFiletypesToString( self._filetype_filter_predicate.GetValue() ) ) )
if self._exclude_deleted:
statements.append( 'excluding previously deleted' )
@ -361,12 +419,9 @@ class FileImportOptions( HydrusSerialisable.SerialisableBase ):
return summary
def CheckReadyToImport( self ) -> bool:
def IsDefault( self ) -> bool:
if self._import_destination_location_context.IsEmpty():
raise HydrusExceptions.FileImportBlockException( 'There is no import destination set in the File Import Options!' )
return self._is_default
def SetDestinationLocationContext( self, location_context: ClientLocation.LocationContext ):
@ -374,6 +429,18 @@ class FileImportOptions( HydrusSerialisable.SerialisableBase ):
self._import_destination_location_context = location_context.Duplicate()
def SetAllowedSpecificFiletypes( self, mimes ) -> None:
mimes = ClientSearch.ConvertSpecificFiletypesToSummary( mimes, only_searchable = False )
self._filetype_filter_predicate = ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_MIME, value = mimes )
def SetIsDefault( self, value: bool ) -> None:
self._is_default = value
def SetPostImportOptions( self, automatic_archive: bool, associate_primary_urls: bool, associate_source_urls: bool ):
self._automatic_archive = automatic_archive

2
hydrus/client/importing/options/PresentationImportOptions.py
View File

@ -246,6 +246,8 @@ class PresentationImportOptions( HydrusSerialisable.SerialisableBase ):
summary = '{}, {}'.format( summary, s )
summary = 'presenting {}'.format( summary )
return summary

4
hydrus/client/importing/options/TagImportOptions.py
View File

@ -661,9 +661,9 @@ class TagImportOptions( HydrusSerialisable.SerialisableBase ):
return self._is_default
def SetDefault( self ):
def SetIsDefault( self, value: bool ):
self._is_default = True
self._is_default = value
def ShouldFetchTagsEvenIfHashKnownAndFileAlreadyInDB( self ):

3
hydrus/client/networking/ClientLocalServerResources.py
View File

@ -40,6 +40,7 @@ from hydrus.client import ClientSearch
from hydrus.client import ClientSearchParseSystemPredicates
from hydrus.client import ClientThreading
from hydrus.client.importing import ClientImportFiles
from hydrus.client.importing.options import FileImportOptions
from hydrus.client.media import ClientMedia
from hydrus.client.metadata import ClientTags
from hydrus.client.networking import ClientNetworkingContexts
@ -1267,7 +1268,7 @@ class HydrusResourceClientAPIRestrictedAddFilesAddFile( HydrusResourceClientAPIR
( os_file_handle, temp_path ) = request.temp_file_info
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'quiet' )
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( FileImportOptions.IMPORT_TYPE_QUIET )
file_import_job = ClientImportFiles.FileImportJob( temp_path, file_import_options )

2
hydrus/core/HydrusConstants.py
View File

@ -80,7 +80,7 @@ options = {}
# Misc
NETWORK_VERSION = 20
SOFTWARE_VERSION = 493
SOFTWARE_VERSION = 494
CLIENT_API_VERSION = 31
SERVER_THUMBNAIL_DIMENSIONS = ( 200, 200 )

2
hydrus/core/HydrusDocumentHandling.py
View File

@ -16,6 +16,8 @@ def GetPDFNumWords( path ):
return None
num_words = None
try:
pass

14
hydrus/core/HydrusPaths.py
View File

@ -493,6 +493,20 @@ def LaunchFile( path, launch_path = None ):
def MakeSureDirectoryExists( path ):
it_exists_already = os.path.exists( path )
if it_exists_already:
if os.path.isdir( path ):
return
else:
raise Exception( 'Sorry, the desired directory "{}" already exists as a normal file!'.format( path ) )
os.makedirs( path, exist_ok = True )
def safe_copy2( source, dest ):

3
hydrus/hydrus_client.py
View File

@ -9,11 +9,12 @@ import locale
try: locale.setlocale( locale.LC_ALL, '' )
except: pass
import sys
try:
import os
import argparse
import sys
from hydrus.core import HydrusBoot

3
hydrus/hydrus_server.py
View File

@ -6,6 +6,8 @@
action = 'start'
import sys
try:
import locale
@ -14,7 +16,6 @@ try:
except: pass
import os
import sys
import threading
from hydrus.core import HydrusBoot

2
hydrus/server/ServerController.py
View File

@ -365,7 +365,7 @@ class Controller( HydrusController.HydrusController ):
self.InitModel()
HydrusData.Print( 'Initialising daemons\u2026' )
HydrusData.Print( 'Initialising workers\u2026' )
self.InitView()

35
hydrus/test/TestClientDB.py
View File

@ -83,8 +83,8 @@ class TestClientDB( unittest.TestCase ):
def test_autocomplete( self ):
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
location_context = ClientLocation.LocationContext.STATICCreateSimple( CC.COMBINED_FILE_SERVICE_KEY )
tag_context = ClientSearch.TagContext( service_key = CC.DEFAULT_LOCAL_TAG_SERVICE_KEY )
@ -366,7 +366,8 @@ class TestClientDB( unittest.TestCase ):
path = os.path.join( HC.STATIC_DIR, 'hydrus.png' )
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
file_import_job = ClientImportFiles.FileImportJob( path, file_import_options )
@ -755,7 +756,8 @@ class TestClientDB( unittest.TestCase ):
path = os.path.join( HC.STATIC_DIR, 'hydrus.png' )
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
file_import_job = ClientImportFiles.FileImportJob( path, file_import_options )
@ -811,7 +813,8 @@ class TestClientDB( unittest.TestCase ):
#
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
file_import_job = ClientImportFiles.FileImportJob( path, file_import_options )
@ -919,7 +922,8 @@ class TestClientDB( unittest.TestCase ):
#
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
file_import_job = ClientImportFiles.FileImportJob( path, file_import_options )
@ -1241,7 +1245,8 @@ class TestClientDB( unittest.TestCase ):
test_files.append( ( 'muh_apng.png', '9e7b8b5abc7cb11da32db05671ce926a2a2b701415d1b2cb77a28deea51010c3', 616956, HC.IMAGE_APNG, 500, 500, { 3133, 1880, 1125, 1800 }, { 27, 47 }, False, None ) )
test_files.append( ( 'muh_gif.gif', '00dd9e9611ebc929bfc78fde99a0c92800bbb09b9d18e0946cea94c099b211c2', 15660, HC.IMAGE_GIF, 329, 302, { 600 }, { 5 }, False, None ) )
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
for ( filename, hex_hash, size, mime, width, height, durations, num_frames, has_audio, num_words ) in test_files:
@ -1292,8 +1297,8 @@ class TestClientDB( unittest.TestCase ):
def test_import_folders( self ):
import_folder_1 = ClientImportLocal.ImportFolder( 'imp 1', path = TestController.DB_DIR, mimes = HC.VIDEO, publish_files_to_popup_button = False )
import_folder_2 = ClientImportLocal.ImportFolder( 'imp 2', path = TestController.DB_DIR, mimes = HC.IMAGES, period = 1200, publish_files_to_popup_button = False )
import_folder_1 = ClientImportLocal.ImportFolder( 'imp 1', path = TestController.DB_DIR, publish_files_to_popup_button = False )
import_folder_2 = ClientImportLocal.ImportFolder( 'imp 2', path = TestController.DB_DIR, period = 1200, publish_files_to_popup_button = False )
#
@ -1368,7 +1373,8 @@ class TestClientDB( unittest.TestCase ):
#
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
file_import_job = ClientImportFiles.FileImportJob( path, file_import_options )
@ -1436,7 +1442,8 @@ class TestClientDB( unittest.TestCase ):
#
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
file_import_job = ClientImportFiles.FileImportJob( path, file_import_options )
@ -1488,7 +1495,8 @@ class TestClientDB( unittest.TestCase ):
path = os.path.join( HC.STATIC_DIR, 'hydrus.png' )
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
file_import_job = ClientImportFiles.FileImportJob( path, file_import_options )
@ -1718,7 +1726,8 @@ class TestClientDB( unittest.TestCase ):
'9e7b8b5abc7cb11da32db05671ce926a2a2b701415d1b2cb77a28deea51010c3' : 'muh_apng.png'
}
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
for ( hash, filename ) in test_files.items():

3
hydrus/test/TestClientDBDuplicates.py
View File

@ -96,7 +96,8 @@ class TestClientDBDuplicates( unittest.TestCase ):
( size, mime, width, height, duration, num_frames, has_audio, num_words ) = ( 65535, HC.IMAGE_JPEG, 640, 480, None, None, False, None )
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
for hash in self._all_hashes:

22
hydrus/test/TestClientDBTags.py
View File

@ -13,6 +13,7 @@ from hydrus.client import ClientSearch
from hydrus.client import ClientServices
from hydrus.client.db import ClientDB
from hydrus.client.importing import ClientImportFiles
from hydrus.client.importing.options import FileImportOptions
from hydrus.client.metadata import ClientTags
from hydrus.test import TestController
@ -769,7 +770,8 @@ class TestClientDBTags( unittest.TestCase ):
path = os.path.join( HC.STATIC_DIR, 'testing', filename )
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
file_import_job = ClientImportFiles.FileImportJob( path, file_import_options )
@ -1057,7 +1059,8 @@ class TestClientDBTags( unittest.TestCase ):
path = os.path.join( HC.STATIC_DIR, 'testing', 'muh_jpg.jpg' )
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
file_import_job = ClientImportFiles.FileImportJob( path, file_import_options )
@ -1155,7 +1158,8 @@ class TestClientDBTags( unittest.TestCase ):
path = os.path.join( HC.STATIC_DIR, 'testing', 'muh_jpg.jpg' )
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
file_import_job = ClientImportFiles.FileImportJob( path, file_import_options )
@ -1254,7 +1258,8 @@ class TestClientDBTags( unittest.TestCase ):
path = os.path.join( HC.STATIC_DIR, 'testing', 'muh_jpg.jpg' )
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
file_import_job = ClientImportFiles.FileImportJob( path, file_import_options )
@ -1355,7 +1360,8 @@ class TestClientDBTags( unittest.TestCase ):
path = os.path.join( HC.STATIC_DIR, 'testing', 'muh_jpg.jpg' )
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
file_import_job = ClientImportFiles.FileImportJob( path, file_import_options )
@ -1455,7 +1461,8 @@ class TestClientDBTags( unittest.TestCase ):
path = os.path.join( HC.STATIC_DIR, 'testing', 'muh_jpg.jpg' )
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
file_import_job = ClientImportFiles.FileImportJob( path, file_import_options )
@ -2568,7 +2575,8 @@ class TestClientDBTags( unittest.TestCase ):
# doing this again tests a very simple add_file
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
for filename in ( 'muh_jpg.jpg', 'muh_png.png', 'muh_apng.png' ):

3
hydrus/test/TestClientMigration.py
View File

@ -173,7 +173,8 @@ class TestMigration( unittest.TestCase ):
( size, mime, width, height, duration, num_frames, has_audio, num_words ) = ( 65535, HC.IMAGE_JPEG, 640, 480, None, None, False, None )
file_import_options = HG.client_controller.new_options.GetDefaultFileImportOptions( 'loud' )
file_import_options = FileImportOptions.FileImportOptions()
file_import_options.SetIsDefault( True )
for i in range( 100 ):

10
hydrus/test/TestClientTags.py
View File

@ -1839,7 +1839,13 @@ class TestTagObjects( unittest.TestCase ):
p = ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_MIME, ( HC.VIDEO_WEBM, HC.IMAGE_GIF ) )
self.assertEqual( p.ToString(), 'system:filetype is webm, gif' )
self.assertEqual( p.ToString(), 'system:filetype is gif, webm' )
self.assertEqual( p.GetNamespace(), 'system' )
self.assertEqual( p.GetTextsAndNamespaces( render_for_user ), [ ( p.ToString(), p.GetNamespace() ) ] )
p = ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_MIME, ( HC.GENERAL_AUDIO, HC.GENERAL_VIDEO ) )
self.assertEqual( p.ToString(), 'system:filetype is audio, video' )
self.assertEqual( p.GetNamespace(), 'system' )
self.assertEqual( p.GetTextsAndNamespaces( render_for_user ), [ ( p.ToString(), p.GetNamespace() ) ] )
@ -1990,7 +1996,7 @@ class TestTagObjects( unittest.TestCase ):
( 'system:limit is 5,000', "system:limit is 5000" ),
( 'system:limit is 100', "system:limit = 100" ),
( 'system:filetype is jpeg', "system:filetype is jpeg" ),
( 'system:filetype is jpeg, png, apng', "system:filetype = image/jpg, image/png, apng" ),
( 'system:filetype is apng, jpeg, png', "system:filetype = image/jpg, image/png, apng" ),
( 'system:sha256 hash is in 3 hashes', "system:hash = abcdef01 abcdef02 abcdef03" ),
( 'system:md5 hash is in 3 hashes', "system:hash = abcdef01 abcdef, abcdef04 md5" ),
( 'system:md5 hash is abcdef01', "system:hash = abcdef01 md5" ),