RepoMirrors
/
hydrus
mirror of https://github.com/hydrusnetwork/hydrus.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Version 67

This commit is contained in:
Hydrus 2013-04-24 16:23:53 -05:00
parent 0f2b4555d6
commit 328a014200
19 changed files with 694 additions and 375 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

37
help/changelog.html
View File

@ -8,6 +8,43 @@
<div class="content">
<h3>changelog</h3>
<ul>
<li><h3>version 67</h3></li>
<ul>
<li>subscription db access improved</li>
<li>subscription 'delete subs seemingly at random' bug fixed</li>
<li>some nice subs help</li>
<li>subs and repos, if changed during a pause, will restart with new changes</li>
<li>subs and repos will automatically pause while their respective dialogs are open</li>
<li>new namespace | regex listctrl in regex dialog, instead of old sctvcp rubbish</li>
<li>new /aa/aa...0 file storage system for client</li>
<li>export and copy files now export writeable files, not read-only</li>
<li>rejiggered daemon db access, improving maintenance reliablity</li>
<li>dumper and 'show in new page' pages now process content updates correctly</li>
</ul>
<li><h3>version 66</h3></li>
<ul>
<li>subscriptions done! works for all normal download types</li>
<li>pause repo sync, pause subs sync</li>
<li>all download sites moved to the new system</li>
<li>lots of small changes to how download code works</li>
<li>pixiv tags is renamed to pixiv tag, since it only does one!</li>
<li>fixed DA</li>
<li>fixed DA again!</li>
<li>fixed DA tag parsing for all the diff types of username re http://help.deviantart.com/106/</li>
<li>fixed pixiv, somewhat</li>
<li>fixed giphy</li>
<li>fixed boorus for gallery_advance_num > 1</li>
<li>boorus reset in db to default</li>
<li>fixed copy files</li>
<li>fixed getmime</li>
<li>a silly parent assignment for A/C meant they were not closing with pages</li>
<li>made the cpu burn for the thumbnail resizer a bit more polite on hdd</li>
<li>repo sync daemon and subs sync daemon combined</li>
<li>some timing adjustments in sync daemon</li>
<li>subscription_type constant -> site_download_type</li>
<li>fixed temp folder being cleared on startup (a read-only issue)</li>
<li>advancedoptions classes now support setinfo</li>
</ul>
<li><h3>version 65</h3></li>
<ul>
<li>added subscriptions dialog</li>

1
help/getting_started_ratings.html
View File

@ -52,7 +52,6 @@
<p>You can search with system:rating, and sort in the normal manner:</p>
<p><img src="ratings_sort.png" /></p>
<p>And that's it! Remote ratings will make this a _little_ more complicated, but not much.</p>
<p class="right"><a href="getting_started_messages.html">Read about messaging ---></a></p>
<p class="right"><a href="index.html">Go back to the index ---></a></p>
</div>
</body>

34
help/getting_started_subscriptions.html Normal file
View File

@ -0,0 +1,34 @@
<html>
<head>
<title>getting started - subscriptions</title>
<link href="hydrus.ico" rel="shortcut icon" />
<link href="style.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div class="content">
<p><a href="getting_started_tags.html"><--- Back to tags</a></p>
<h3>what are subs?</h3>
<p>Subscriptions are a way of telling the client to regularly check a particular web search. Any new files will be downloaded and imported, and their tags optionally parsed.</p>
<p>You can set up a subscription for any of the gallery websites in the normal <i>new page->download->gallery</i> menu.</p>
<p>Here's the dialog, which is under <i>services->manage subscriptions</i>:</p>
<p><img src="subs_dialog.png" /></p>
<p>Here I have one subscription for Deviant Art, for the artist ChaoyuanXu. The client is set to check this artist for new files to import every seven days, and it will parse the respective creator and title tags and send them to the tag repository called 'public tag repo'.</p>
<p>The subscription synchronisation daemon (the subroutine that checks and imports from the websites) works very much like the one that synchronises hydrus repositories. It works in the background, and reports its current status similarly. When it is time for it to do some work, it will put a little text in the client's statusbar, like so:</p>
<p><img src="subs_status.png" /></p>
<p>You don't really have to care about this all that much; it just lets you know what it is doing.</p>
<p>Errors will be recovered from as gracefully as possible, and details written to the log. The subs daemon will retry the next day.</p>
<p>Here's the result of the subscription I set up above:</p>
<p><img src="subs_import_done.png" /></p>
<p>It took about two minutes to download all that, and it all happened quietly in the background. Notice the 146 pending tags, up top.</p>
<h3>how could this possibly go wrong?</h3>
<p>This is quite a powerful tool, and if you are silly, you will end up spamming a server and likely upsetting someone or breaking something.</p>
<p>To initialise a subscription, the daemon will parse every single gallery and image page for that particular search. This is fine for the example above, which had 4 gallery pages and 73 image pages, but the search "short hair" on safebooru has about 6,400 gallery pages encompassing >250,000 results! Trying a search like that will take a tremendous amount of time for you and cause a non-trivial CPU and data hit to their server.</p>
<p><i>Remember: If you are going to scrape another's site, be polite about it!</i></p>
<p>So, I advise you start with artist searches to begin with. These usually top out at about 1,000 files, and hence don't take all that long to do. Once you are more confident, try doing multiple-tag queries. I suggest you leave simple single-tag queries for the manual download page, where you can hit 'that's enough' after ten or twenty pages.</p>
<h3>help! it won't stop!</h3>
<p>If you <i>do</i> put in a huge search, and the 'found x new files for subscription y' message is climbing terrifyingly higher and higher with no end in sight, you can pause the subscriptions daemon with <i>services->pause->subscriptions synchronisation</i>.</p>
<p>This will give you a breather to edit your subscriptions in the dialog. Just unpause to continue with your new subs.</p>
<p class="right"><a href="index.html">Go back to the index ---></a></p>
</div>
</body>
</html>

1
help/getting_started_tags.html
View File

@ -43,6 +43,7 @@
<li>feel good about myself</li>
</ul>
<p class="right"><a href="getting_started_ratings.html">Read about ratings ---></a></p>
<p class="right"><a href="getting_started_subscriptions.html">Read about subscriptions ---></a></p>
<p class="right"><a href="index.html">Go back to the index ---></a></p>
</div>
</body>

79
help/index.html
View File

@ -1,40 +1,41 @@
<html>
<head>
<title>hydrus help</title>
<link href="hydrus.ico" rel="shortcut icon" />
<link href="style.css" rel="stylesheet" type="text/css" />
</head>
<body>
<a class="screenshot" href="client_empty.png" title="the main screen, just after you start up"><img src="client_empty_small.png" /></a>
<a class="screenshot" href="client_autism.png" title="an example search"><img src="client_autism_small.png" /></a>
<a class="screenshot" href="lib_gc.png" title="a well-tagged webcomic, sorted and viewed by chapter"><img src="lib_gc_small.png" /></a>
<a class="screenshot" href="client_fullscreen.png" title="reading the webcomic in fullscreen"><img src="client_fullscreen_small.png" /></a>
<a class="screenshot" href="lib_rec.png" title="file repository view, with some files local and some not"><img src="lib_rec_small.png" /></a>
<a class="screenshot" href="lib_party_hard.png" title="now with swf support!"><img src="lib_party_hard_small.png" /></a>
<a class="screenshot" href="client_auto.png" title="an example of tag-autocomplete, which only displays results applicable to the current query"><img src="client_auto_small.png" /></a>
<h3>hydrus help</h3>
<p>Although the hydrus software's interface attempts to be simple, its underlying concepts are not. Please read the introduction and skim the getting started guide at the least, and you'll probably want to check out the access keys section to get started with my server.</p>
<ul>
<li><a href="introduction.html">introduction and statement of principles</a></li>
<li><a href="getting_started_files.html">getting started with files</a></li>
<li><a href="getting_started_tags.html">getting started with tags</a></li>
<li><a href="getting_started_ratings.html">getting started with ratings</a></li>
<li><a href="getting_started_messages.html">getting started with messages</a></li>
<li><a href="registration_keys.html">registering new accounts</a></li>
<li><a href="access_keys.html">access keys to my server's services</a></li>
<li><a href="tagging_schema.html">thoughts on a public tagging schema</a></li>
<li><a href="advanced.html">advanced usage</a></li>
<li><a href="privacy.html">privacy</a></li>
<li><a href="server.html">setting up your own server</a></li>
<li><a href="running_from_source.html">running a client or server from source</a></li>
<li><a href="contact.html">developer contact and links</a></li>
<li><a href="future.html">ideas for the future</a></li>
<li><a href="updates.html">how the repositories synchronise</a></li>
<li><a href="depots.html">how the message depots work</a></li>
<li><a href="db_diagrams.html">database diagrams</a></li>
<li><a href="faq.html">faq</a></li>
<li><a href="glossary.html">glossary</a></li>
<li><a href="changelog.html">changelog</a></li>
</ul>
</body>
<html>
<head>
<title>hydrus help</title>
<link href="hydrus.ico" rel="shortcut icon" />
<link href="style.css" rel="stylesheet" type="text/css" />
</head>
<body>
<a class="screenshot" href="client_empty.png" title="the main screen, just after you start up"><img src="client_empty_small.png" /></a>
<a class="screenshot" href="client_autism.png" title="an example search"><img src="client_autism_small.png" /></a>
<a class="screenshot" href="lib_gc.png" title="a well-tagged webcomic, sorted and viewed by chapter"><img src="lib_gc_small.png" /></a>
<a class="screenshot" href="client_fullscreen.png" title="reading the webcomic in fullscreen"><img src="client_fullscreen_small.png" /></a>
<a class="screenshot" href="lib_rec.png" title="file repository view, with some files local and some not"><img src="lib_rec_small.png" /></a>
<a class="screenshot" href="lib_party_hard.png" title="now with swf support!"><img src="lib_party_hard_small.png" /></a>
<a class="screenshot" href="client_auto.png" title="an example of tag-autocomplete, which only displays results applicable to the current query"><img src="client_auto_small.png" /></a>
<h3>hydrus help</h3>
<p>Although the hydrus software's interface attempts to be simple, its underlying concepts are not. Please read the introduction and skim the getting started guide at the least, and you'll probably want to check out the access keys section to get started with my server.</p>
<ul>
<li><a href="introduction.html">introduction and statement of principles</a></li>
<li><a href="getting_started_files.html">getting started with files</a></li>
<li><a href="getting_started_tags.html">getting started with tags</a></li>
<li><a href="getting_started_ratings.html">getting started with ratings</a></li>
<li><a href="getting_started_subscriptions.html">getting started with subscriptions</a></li>
<li><a href="getting_started_messages.html">getting started with messages</a></li>
<li><a href="registration_keys.html">registering new accounts</a></li>
<li><a href="access_keys.html">access keys to my server's services</a></li>
<li><a href="tagging_schema.html">thoughts on a public tagging schema</a></li>
<li><a href="advanced.html">advanced usage</a></li>
<li><a href="privacy.html">privacy</a></li>
<li><a href="server.html">setting up your own server</a></li>
<li><a href="running_from_source.html">running a client or server from source</a></li>
<li><a href="contact.html">developer contact and links</a></li>
<li><a href="future.html">ideas for the future</a></li>
<li><a href="updates.html">how the repositories synchronise</a></li>
<li><a href="depots.html">how the message depots work</a></li>
<li><a href="db_diagrams.html">database diagrams</a></li>
<li><a href="faq.html">faq</a></li>
<li><a href="glossary.html">glossary</a></li>
<li><a href="changelog.html">changelog</a></li>
</ul>
</body>
</html>

BIN
help/subs_dialog.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 33 KiB

BIN
help/subs_import_done.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 430 KiB

BIN
help/subs_status.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.4 KiB

48
include/ClientConstants.py
View File

@ -270,6 +270,42 @@ def GenerateMultipartFormDataCTAndBodyFromDict( fields ):
return m.get()
def GetAllFileHashes():
file_hashes = set()
for path in IterateAllFilePaths():
( base, filename ) = os.path.split( path )
try:
( hash_encoded, ext ) = filename.split( '.', 1 )
file_hashes.add( hash_encoded.decode( 'hex' ) )
except: continue
return file_hashes
def GetAllThumbnailHashes():
thumbnail_hashes = set()
for path in IterateAllThumbnailPaths():
( base, filename ) = os.path.split( path )
if not filename.endswith( '_resized' ):
try: thumbnail_hashes.add( filename.decode( 'hex' ) )
except: continue
return thumbnail_hashes
def GetFilePath( hash, mime ):
hash_encoded = hash.encode( 'hex' )
@ -322,9 +358,11 @@ def IterateAllFilePaths():
for ( one, two ) in itertools.product( hex_chars, hex_chars ):
next_paths = dircache.listdir( HC.CLIENT_FILES_DIR + os.path.sep + one + two )
dir = HC.CLIENT_FILES_DIR + os.path.sep + one + two
for path in next_paths: yield path
next_paths = dircache.listdir( dir )
for path in next_paths: yield dir + os.path.sep + path
def IterateAllThumbnailPaths():
@ -333,9 +371,11 @@ def IterateAllThumbnailPaths():
for ( one, two ) in itertools.product( hex_chars, hex_chars ):
next_paths = dircache.listdir( HC.CLIENT_THUMBNAIL_DIR + os.path.sep + one + two )
dir = HC.CLIENT_THUMBNAILS_DIR + os.path.sep + one + two
for path in next_paths: yield path
next_paths = dircache.listdir( dir )
for path in next_paths: yield dir + os.path.sep + path
def MediaIntersectCDPPTagServiceIdentifiers( media, service_identifier ):

24
include/ClientController.py
View File

@ -233,9 +233,7 @@ class Controller( wx.App ):
def ProcessServerRequest( self, *args, **kwargs ): return self._db.ProcessRequest( *args, **kwargs )
def Read( self, action, *args, **kwargs ):
self._last_idle_time = int( time.time() )
def _Read( self, action, *args, **kwargs ):
if action == 'options': return self._options
elif action == 'tag_service_precedence': return self._tag_service_precedence
@ -244,6 +242,15 @@ class Controller( wx.App ):
else: return self._db.Read( action, HC.HIGH_PRIORITY, *args, **kwargs )
def Read( self, action, *args, **kwargs ):
self._last_idle_time = int( time.time() )
return self._Read( action, *args, **kwargs )
def ReadDaemon( self, action, *args, **kwargs ): return self._Read( action, *args, **kwargs )
def SetSplashText( self, text ):
self._splash.SetText( text )
@ -262,15 +269,16 @@ class Controller( wx.App ):
def _Write( self, action, priority, *args, **kwargs ): self._db.Write( action, priority, *args, **kwargs )
def Write( self, action, *args, **kwargs ):
self._last_idle_time = int( time.time() )
self._db.Write( action, HC.HIGH_PRIORITY, *args, **kwargs )
self._Write( action, HC.HIGH_PRIORITY, *args, **kwargs )
def WriteLowPriority( self, action, *args, **kwargs ):
self._db.Write( action, HC.LOW_PRIORITY, *args, **kwargs )
def WriteDaemon( self, action, *args, **kwargs ): self._Write( action, HC.LOW_PRIORITY, *args, **kwargs )
def WriteLowPriority( self, action, *args, **kwargs ): self._Write( action, HC.LOW_PRIORITY, *args, **kwargs )

360
include/ClientDB.py
View File

@ -35,9 +35,9 @@ class FileDB():
hash_id = self._GetHashId( c, hash )
thumbnail_path_to = HC.CLIENT_THUMBNAILS_DIR + os.path.sep + hash.encode( 'hex' )
thumbnail_path = CC.GetThumbnailPath( hash )
with open( thumbnail_path_to, 'wb' ) as f: f.write( thumbnail )
with open( thumbnail_path, 'wb' ) as f: f.write( thumbnail )
phash = HydrusImageHandling.GeneratePerceptualHash( thumbnail )
@ -67,12 +67,14 @@ class FileDB():
mime = self._GetMime( c, self._local_file_service_id, hash_id )
path_from = HC.CLIENT_FILES_DIR + os.path.sep + hash.encode( 'hex' ) + HC.mime_ext_lookup[ mime ]
path_from = CC.GetFilePath( hash, mime )
path_to = export_path + os.path.sep + hash.encode( 'hex' ) + HC.mime_ext_lookup[ mime ]
shutil.copy( path_from, path_to )
os.chmod( path_to, stat.S_IWRITE )
paths.append( path_to )
except Exception as e: error_messages.add( unicode( e ) )
@ -115,12 +117,14 @@ class FileDB():
mime = self._GetMime( c, self._local_file_service_id, hash_id )
path_from = HC.CLIENT_FILES_DIR + os.path.sep + hash.encode( 'hex' ) + HC.mime_ext_lookup[ mime ]
path_from = CC.GetFilePath( hash, mime )
path_to = export_path + os.path.sep + hash.encode( 'hex' ) + HC.mime_ext_lookup[ mime ]
shutil.copy( path_from, path_to )
os.chmod( path_to, stat.S_IWRITE )
if cancel_event.isSet(): break
except Exception as e: error_messages.add( unicode( e ) )
@ -158,7 +162,7 @@ class FileDB():
with self._hashes_to_mimes_lock: mime = self._hashes_to_mimes[ hash ]
with open( HC.CLIENT_FILES_DIR + os.path.sep + hash.encode( 'hex' ) + HC.mime_ext_lookup[ mime ], 'rb' ) as f: file = f.read()
with open( CC.GetFilePath( hash, mime ), 'rb' ) as f: file = f.read()
except MemoryError: print( 'Memory error!' )
except: raise Exception( 'Could not find that file!' )
@ -226,34 +230,30 @@ class FileDB():
def _GetThumbnail( self, hash, full_size = False ):
if full_size:
path = CC.GetThumbnailPath( hash )
if not full_size:
path_to = HC.CLIENT_THUMBNAILS_DIR + os.path.sep + hash.encode( 'hex' )
fullsize_path = path
with open( path_to, 'rb' ) as f: thumbnail = f.read()
path = fullsize_path + '_resized'
else:
path_to = HC.CLIENT_THUMBNAILS_DIR + os.path.sep + hash.encode( 'hex' ) + '_resized'
if os.path.exists( path_to ):
if not os.path.exists( path ):
with open( path_to, 'rb' ) as f: thumbnail = f.read()
else:
path_to_full = HC.CLIENT_THUMBNAILS_DIR + os.path.sep + hash.encode( 'hex' )
with open( path_to_full, 'rb' ) as f: thumbnail_full = f.read()
with open( fullsize_path, 'rb' ) as f: thumbnail_full = f.read()
thumbnail_dimensions = self._options[ 'thumbnail_dimensions' ]
thumbnail = HydrusImageHandling.GenerateThumbnailFileFromFile( thumbnail_full, thumbnail_dimensions )
with open( path_to, 'wb' ) as f: f.write( thumbnail )
with open( path, 'wb' ) as f: f.write( thumbnail )
return thumbnail
with open( path, 'rb' ) as f: thumbnail = f.read()
return thumbnail
@ -1724,30 +1724,13 @@ class ServiceDB( FileDB, MessageDB, TagDB, RatingDB ):
deletee_hashes = set( self._GetHashes( c, deletee_hash_ids ) )
cached_filenames = dircache.listdir( HC.CLIENT_FILES_DIR )
local_files_hashes = set()
hashes_to_filenames = {}
for filename in cached_filenames:
try:
( hash, ext ) = filename.split( '.' )
hash = hash.decode( 'hex' )
local_files_hashes.add( hash ) # this try ... except is for weird files that might have got into the directory by accident
hashes_to_filenames[ hash ] = filename
except: pass
local_files_hashes = CC.GetAllFileHashes()
for hash in local_files_hashes & deletee_hashes:
path = HC.CLIENT_FILES_DIR + os.path.sep + hashes_to_filenames[ hash ]
with self._hashes_to_mimes_lock: mime = self._hashes_to_mimes[ hash ]
path = CC.GetFilePath( hash, mime )
os.chmod( path, stat.S_IWRITE )
@ -1764,26 +1747,15 @@ class ServiceDB( FileDB, MessageDB, TagDB, RatingDB ):
c.execute( 'DELETE FROM perceptual_hashes WHERE hash_id IN ' + HC.SplayListForDB( perceptual_deletees ) + ';' )
all_thumbnail_filenames = dircache.listdir( HC.CLIENT_THUMBNAILS_DIR )
thumbnails_i_have = set()
for filename in all_thumbnail_filenames:
if not filename.endswith( '_resized' ):
try: thumbnails_i_have.add( filename.decode( 'hex' ) ) # this try ... except is for weird files that might have got into the directory by accident
except: pass
local_thumbnail_hashes = CC.GetAllThumbnailHashes()
hashes = set( self._GetHashes( c, hash_ids ) )
thumbnail_deletees = thumbnails_i_have - hashes
thumbnail_deletees = local_thumbnail_hashes - hashes
for hash in thumbnail_deletees:
path = HC.CLIENT_THUMBNAILS_DIR + os.path.sep + hash.encode( 'hex' )
path = CC.GetThumbnailPath( hash )
resized_path = path + '_resized'
os.remove( path )
@ -2960,7 +2932,7 @@ class ServiceDB( FileDB, MessageDB, TagDB, RatingDB ):
elif info_type == HC.SERVICE_INFO_NUM_THUMBNAILS: result = c.execute( 'SELECT COUNT( * ) FROM files_info WHERE service_id = ? AND mime IN ' + HC.SplayListForDB( HC.MIMES_WITH_THUMBNAILS ) + ';', ( service_id, ) ).fetchone()
elif info_type == HC.SERVICE_INFO_NUM_THUMBNAILS_LOCAL:
thumbnails_i_have = { path.decode( 'hex' ) for path in dircache.listdir( HC.CLIENT_THUMBNAILS_DIR ) if not path.endswith( '_resized' ) }
thumbnails_i_have = CC.GetAllThumbnailHashes()
hash_ids = [ hash_id for ( hash_id, ) in c.execute( 'SELECT hash_id FROM files_info WHERE mime IN ' + HC.SplayListForDB( HC.MIMES_WITH_THUMBNAILS ) + ' AND service_id = ?;', ( service_id, ) ) ]
@ -3141,18 +3113,9 @@ class ServiceDB( FileDB, MessageDB, TagDB, RatingDB ):
def _GetSubscriptions( self, c ):
result = c.execute( 'SELECT subscriptions FROM subscriptions;', ).fetchone()
subscriptions = [ ( site_download_type, name, query_type, query, frequency_type, frequency_number, dict( advanced_tag_options ), advanced_import_options, last_checked, url_cache ) for ( site_download_type, name, ( query_type, query, frequency_type, frequency_number, advanced_tag_options, advanced_import_options, last_checked, url_cache ) ) in c.execute( 'SELECT site_download_type, name, info FROM subscriptions;', ) ]
if result is None: return []
else:
( subscriptions, ) = result
# dict( advanced_tag_options ) is a db yaml storage bug
subscriptions = [ ( site_download_type, name, query_type, query, frequency_type, frequency_number, dict( advanced_tag_options ), advanced_import_options, last_checked, url_cache ) for ( site_download_type, name, query_type, query, frequency_type, frequency_number, advanced_tag_options, advanced_import_options, last_checked, url_cache ) in subscriptions ]
return subscriptions
return subscriptions
def _GetTagServicePrecedence( self, c ):
@ -3325,7 +3288,7 @@ class ServiceDB( FileDB, MessageDB, TagDB, RatingDB ):
timestamp = int( time.time() )
dest_path = HC.CLIENT_FILES_DIR + os.path.sep + hash.encode( 'hex' ) + HC.mime_ext_lookup[ mime ]
dest_path = CC.GetFilePath( hash, mime )
if not os.path.exists( dest_path ):
@ -3338,15 +3301,15 @@ class ServiceDB( FileDB, MessageDB, TagDB, RatingDB ):
thumbnail = HydrusImageHandling.GenerateThumbnailFileFromFile( file, HC.UNSCALED_THUMBNAIL_DIMENSIONS )
thumbnail_path_to = HC.CLIENT_THUMBNAILS_DIR + os.path.sep + hash.encode( 'hex' )
thumbnail_path = CC.GetThumbnailPath( hash )
with open( thumbnail_path_to, 'wb' ) as f: f.write( thumbnail )
with open( thumbnail_path, 'wb' ) as f: f.write( thumbnail )
thumbnail_resized = HydrusImageHandling.GenerateThumbnailFileFromFile( thumbnail, self._options[ 'thumbnail_dimensions' ] )
thumbnail_resized_path_to = HC.CLIENT_THUMBNAILS_DIR + os.path.sep + hash.encode( 'hex' ) + '_resized'
thumbnail_resized_path = thumbnail_path + '_resized'
with open( thumbnail_resized_path_to, 'wb' ) as f: f.write( thumbnail_resized )
with open( thumbnail_resized_path, 'wb' ) as f: f.write( thumbnail_resized )
phash = HydrusImageHandling.GeneratePerceptualHash( thumbnail )
@ -3801,14 +3764,28 @@ class ServiceDB( FileDB, MessageDB, TagDB, RatingDB ):
c.execute( 'INSERT INTO pixiv_account ( pixiv_id, password ) VALUES ( ?, ? );', ( pixiv_id, password ) )
def _SetSubscription( self, c, subscription ):
( site_download_type, name, query_type, query, frequency_type, frequency_number, advanced_tag_options, advanced_import_options, last_checked, url_cache ) = subscription
info = [ query_type, query, frequency_type, frequency_number, advanced_tag_options.items(), advanced_import_options, last_checked, url_cache ]
c.execute( 'DELETE FROM subscriptions WHERE site_download_type = ? AND name = ?;', ( site_download_type, name ) )
c.execute( 'INSERT INTO subscriptions ( site_download_type, name, info ) VALUES ( ?, ?, ? );', ( site_download_type, name, info ) )
def _SetSubscriptions( self, c, subscriptions ):
HC.repos_or_subs_changed = True
inserts = [ ( site_download_type, name, [ query_type, query, frequency_type, frequency_number, advanced_tag_options.items(), advanced_import_options, last_checked, url_cache ] ) for ( site_download_type, name, query_type, query, frequency_type, frequency_number, advanced_tag_options, advanced_import_options, last_checked, url_cache ) in subscriptions ]
c.execute( 'DELETE FROM subscriptions;' )
# advanced_tag_options.items() is a db yaml storage bug
subscriptions = [ ( site_download_type, name, query_type, query, frequency_type, frequency_number, advanced_tag_options.items(), advanced_import_options, last_checked, url_cache ) for ( site_download_type, name, query_type, query, frequency_type, frequency_number, advanced_tag_options, advanced_import_options, last_checked, url_cache ) in subscriptions ]
c.executemany( 'INSERT INTO subscriptions ( site_download_type, name, info ) VALUES ( ?, ?, ? );', inserts )
c.execute( 'INSERT INTO subscriptions ( subscriptions ) VALUES ( ? );', ( subscriptions, ) )
self.pub( 'notify_new_subscriptions' )
def _SetTagServicePrecedence( self, c, service_identifiers ):
@ -4118,6 +4095,8 @@ class ServiceDB( FileDB, MessageDB, TagDB, RatingDB ):
def _UpdateServices( self, c, edit_log ):
HC.repos_or_subs_changed = True
recalc_active_mappings = False
for ( action, details ) in edit_log:
@ -4446,10 +4425,13 @@ class DB( ServiceDB ):
if os.path.exists( temp_dir ): shutil.rmtree( temp_dir, onerror = make_temp_files_deletable )
except: pass
try:
if not os.path.exists( temp_dir ): os.mkdir( temp_dir )
except: pass
# clean up if last connection closed badly
@ -4604,6 +4586,19 @@ class DB( ServiceDB ):
if not os.path.exists( HC.CLIENT_FILES_DIR ): os.mkdir( HC.CLIENT_FILES_DIR )
if not os.path.exists( HC.CLIENT_THUMBNAILS_DIR ): os.mkdir( HC.CLIENT_THUMBNAILS_DIR )
hex_chars = '0123456789abcdef'
for ( one, two ) in itertools.product( hex_chars, hex_chars ):
dir = HC.CLIENT_FILES_DIR + os.path.sep + one + two
if not os.path.exists( dir ): os.mkdir( dir )
dir = HC.CLIENT_THUMBNAILS_DIR + os.path.sep + one + two
if not os.path.exists( dir ): os.mkdir( dir )
( db, c ) = self._GetDBCursor()
c.execute( 'PRAGMA auto_vacuum = 0;' ) # none
@ -4753,7 +4748,7 @@ class DB( ServiceDB ):
c.execute( 'CREATE TABLE statuses ( status_id INTEGER PRIMARY KEY, status TEXT );' )
c.execute( 'CREATE UNIQUE INDEX statuses_status_index ON statuses ( status );' )
c.execute( 'CREATE TABLE subscriptions ( subscriptions TEXT_YAML );' )
c.execute( 'CREATE TABLE subscriptions ( site_download_type INTEGER, name TEXT, info TEXT_YAML, PRIMARY KEY( site_download_type, name ) );' )
c.execute( 'CREATE TABLE tag_service_precedence ( service_id INTEGER PRIMARY KEY REFERENCES services ON DELETE CASCADE, precedence INTEGER );' )
@ -4930,9 +4925,9 @@ class DB( ServiceDB ):
if resize_thumbs:
thumbnail_paths = [ path for path in dircache.listdir( HC.CLIENT_THUMBNAILS_DIR ) if path.endswith( '_resized' ) ]
thumbnail_paths = [ path for path in CC.IterateAllThumbnailPaths() if path.endswith( '_resized' ) ]
for path in thumbnail_paths: os.remove( HC.CLIENT_THUMBNAILS_DIR + os.path.sep + path )
for path in thumbnail_paths: os.remove( path )
self.pub( 'thumbnail_resize' )
@ -5111,6 +5106,89 @@ class DB( ServiceDB ):
c.execute( 'UPDATE options SET options = ?;', ( self._options, ) )
if version < 67:
result = c.execute( 'SELECT subscriptions FROM subscriptions;' ).fetchone()
if result is None: subscriptions = []
else: ( subscriptions, ) = result
c.execute( 'DROP TABLE subscriptions;' )
c.execute( 'CREATE TABLE subscriptions ( site_download_type INTEGER, name TEXT, info TEXT_YAML, PRIMARY KEY( site_download_type, name ) );' )
inserts = [ ( site_download_type, name, [ query_type, query, frequency_type, frequency_number, advanced_tag_options, advanced_import_options, last_checked, url_cache ] ) for ( site_download_type, name, query_type, query, frequency_type, frequency_number, advanced_tag_options, advanced_import_options, last_checked, url_cache ) in subscriptions ]
c.executemany( 'INSERT INTO subscriptions ( site_download_type, name, info ) VALUES ( ?, ?, ? );', inserts )
#
wx.GetApp().SetSplashText( 'creating new db directories' )
hex_chars = '0123456789abcdef'
for ( one, two ) in itertools.product( hex_chars, hex_chars ):
dir = HC.CLIENT_FILES_DIR + os.path.sep + one + two
if not os.path.exists( dir ): os.mkdir( dir )
dir = HC.CLIENT_THUMBNAILS_DIR + os.path.sep + one + two
if not os.path.exists( dir ): os.mkdir( dir )
wx.GetApp().SetSplashText( 'generating file cache' )
filenames = dircache.listdir( HC.CLIENT_FILES_DIR )
i = 1
for filename in filenames:
try:
source_path = HC.CLIENT_FILES_DIR + os.path.sep + filename
first_two_chars = filename[:2]
destination_path = HC.CLIENT_FILES_DIR + os.path.sep + first_two_chars + os.path.sep + filename
shutil.move( source_path, destination_path )
except: continue
i += 1
if i % 100 == 0: wx.GetApp().SetSplashText( 'moving files - ' + HC.ConvertIntToPrettyString( i ) + '/' + HC.ConvertIntToPrettyString( len( filenames ) ) )
wx.GetApp().SetSplashText( 'generating thumbnail cache' )
filenames = dircache.listdir( HC.CLIENT_THUMBNAILS_DIR )
i = 1
for filename in filenames:
try:
source_path = HC.CLIENT_THUMBNAILS_DIR + os.path.sep + filename
first_two_chars = filename[:2]
destination_path = HC.CLIENT_THUMBNAILS_DIR + os.path.sep + first_two_chars + os.path.sep + filename
shutil.move( source_path, destination_path )
except: continue
i += 1
if i % 100 == 0: wx.GetApp().SetSplashText( 'moving thumbnails - ' + HC.ConvertIntToPrettyString( i ) + '/' + HC.ConvertIntToPrettyString( len( filenames ) ) )
unknown_account = CC.GetUnknownAccount()
unknown_account.MakeStale()
@ -6241,7 +6319,7 @@ class DB( ServiceDB ):
def DAEMONDownloadFiles( self ):
all_downloads = self.Read( 'all_downloads', HC.LOW_PRIORITY )
all_downloads = wx.GetApp().ReadDaemon( 'all_downloads' )
num_downloads = sum( [ len( hashes ) for ( service_identifier, hashes ) in all_downloads.items() ] )
@ -6249,7 +6327,7 @@ class DB( ServiceDB ):
try:
try: file_repository = self.Read( 'service', HC.LOW_PRIORITY, service_identifier )
try: file_repository = wx.GetApp().ReadDaemon( 'service', service_identifier )
except: continue
HC.pubsub.pub( 'downloads_status', HC.ConvertIntToPrettyString( num_downloads ) + ' file downloads' )
@ -6270,7 +6348,7 @@ class DB( ServiceDB ):
HC.pubsub.pub( 'downloads_status', HC.ConvertIntToPrettyString( num_downloads ) + ' file downloads' )
self.Write( 'import_file', HC.LOW_PRIORITY, file )
wx.GetApp().WriteDaemon( 'import_file', file )
HC.pubsub.pub( 'content_updates_data', [ HC.ContentUpdate( HC.CONTENT_UPDATE_ADD, HC.LOCAL_FILE_SERVICE_IDENTIFIER, ( hash, ) ) ] )
HC.pubsub.pub( 'content_updates_gui', [ HC.ContentUpdate( HC.CONTENT_UPDATE_ADD, HC.LOCAL_FILE_SERVICE_IDENTIFIER, ( hash, ) ) ] )
@ -6290,19 +6368,19 @@ class DB( ServiceDB ):
def DAEMONDownloadThumbnails( self ):
service_identifiers = self.Read( 'service_identifiers', HC.LOW_PRIORITY, ( HC.FILE_REPOSITORY, ) )
service_identifiers = wx.GetApp().ReadDaemon( 'service_identifiers', ( HC.FILE_REPOSITORY, ) )
thumbnail_hashes_i_have = { path.decode( 'hex' ) for path in dircache.listdir( HC.CLIENT_THUMBNAILS_DIR ) if not path.endswith( '_resized' ) }
thumbnail_hashes_i_have = CC.GetAllThumbnailHashes()
for service_identifier in service_identifiers:
thumbnail_hashes_i_should_have = self.Read( 'thumbnail_hashes_i_should_have', HC.LOW_PRIORITY, service_identifier )
thumbnail_hashes_i_should_have = wx.GetApp().ReadDaemon( 'thumbnail_hashes_i_should_have', service_identifier )
thumbnail_hashes_i_need = list( thumbnail_hashes_i_should_have - thumbnail_hashes_i_have )
if len( thumbnail_hashes_i_need ) > 0:
try: file_repository = self.Read( 'service', HC.LOW_PRIORITY, service_identifier )
try: file_repository = wx.GetApp().ReadDaemon( 'service', service_identifier )
except: continue
if file_repository.CanDownload():
@ -6323,7 +6401,7 @@ class DB( ServiceDB ):
wx.GetApp().WaitUntilGoodTimeToUseGUIThread()
self.Write( 'thumbnails', HC.LOW_PRIORITY, thumbnails )
wx.GetApp().WriteDaemon( 'thumbnails', thumbnails )
self.pub( 'add_thumbnail_count', service_identifier, len( thumbnails ) )
@ -6340,19 +6418,17 @@ class DB( ServiceDB ):
def DAEMONFlushServiceUpdates( self, update_log ): self.Write( 'service_updates', HC.HIGH_PRIORITY, update_log )
def DAEMONFlushServiceUpdates( self, update_log ): wx.GetApp().WriteDaemon( 'service_updates', update_log )
def DAEMONResizeThumbnails( self ):
all_thumbnail_paths = dircache.listdir( HC.CLIENT_THUMBNAILS_DIR )
all_thumbnail_paths = [ path for path in CC.IterateAllThumbnailPaths() ]
full_size_thumbnail_paths = { path for path in all_thumbnail_paths if not path.endswith( '_resized' ) }
resized_thumbnail_paths = { path for path in all_thumbnail_paths if path.endswith( '_resized' ) }
thumbnail_paths_to_render = full_size_thumbnail_paths.difference( resized_thumbnail_paths )
thumbnail_paths_to_render = list( thumbnail_paths_to_render )
thumbnail_paths_to_render = list( full_size_thumbnail_paths.difference( resized_thumbnail_paths ) )
random.shuffle( thumbnail_paths_to_render )
@ -6364,13 +6440,13 @@ class DB( ServiceDB ):
try:
with open( HC.CLIENT_THUMBNAILS_DIR + os.path.sep + thumbnail_path, 'rb' ) as f: thumbnail = f.read()
with open( thumbnail_path, 'rb' ) as f: thumbnail = f.read()
thumbnail_resized = HydrusImageHandling.GenerateThumbnailFileFromFile( thumbnail, self._options[ 'thumbnail_dimensions' ] )
thumbnail_resized_path_to = thumbnail_path + '_resized'
thumbnail_resized_path = thumbnail_path + '_resized'
with open( HC.CLIENT_THUMBNAILS_DIR + os.path.sep + thumbnail_resized_path_to, 'wb' ) as f: f.write( thumbnail_resized )
with open( thumbnail_resized_path, 'wb' ) as f: f.write( thumbnail_resized )
except: print( traceback.format_exc() )
@ -6392,7 +6468,7 @@ class DB( ServiceDB ):
def DAEMONSynchroniseAccounts( self ):
services = self.Read( 'services', HC.LOW_PRIORITY, HC.RESTRICTED_SERVICES )
services = wx.GetApp().ReadDaemon( 'services', HC.RESTRICTED_SERVICES )
do_notify = False
@ -6432,7 +6508,7 @@ class DB( ServiceDB ):
def DAEMONSynchroniseMessages( self ):
service_identifiers = self.Read( 'service_identifiers', HC.LOW_PRIORITY, ( HC.MESSAGE_DEPOT, ) )
service_identifiers = wx.GetApp().ReadDaemon( 'service_identifiers', ( HC.MESSAGE_DEPOT, ) )
for service_identifier in service_identifiers:
@ -6442,7 +6518,7 @@ class DB( ServiceDB ):
service_type = service_identifier.GetType()
try: service = self.Read( 'service', HC.LOW_PRIORITY, service_identifier )
try: service = wx.GetApp().ReadDaemon( 'service', service_identifier )
except: continue
if service.CanCheck():
@ -6463,9 +6539,9 @@ class DB( ServiceDB ):
connection.Post( 'contact', public_key = public_key )
self.Write( 'contact_associated', HC.HIGH_PRIORITY, service_identifier )
wx.GetApp().WriteDaemon( 'contact_associated', service_identifier )
service = self.Read( 'service', HC.LOW_PRIORITY, service_identifier )
service = wx.GetApp().ReadDaemon( 'service', service_identifier )
contact = service.GetContact()
@ -6493,7 +6569,7 @@ class DB( ServiceDB ):
new_last_check = int( time.time() ) - 5
self.Write( 'message_info_since', HC.LOW_PRIORITY, service_identifier, message_keys, decrypted_statuses, new_last_check )
wx.GetApp().WriteDaemon( 'message_info_since', service_identifier, message_keys, decrypted_statuses, new_last_check )
if len( message_keys ) > 0: HC.pubsub.pub( 'log_message', 'synchronise messages daemon', 'checked ' + service_identifier.GetName() + ' up to ' + HC.ConvertTimestampToPrettyTime( new_last_check ) + ', finding ' + str( len( message_keys ) ) + ' new messages' )
@ -6504,7 +6580,7 @@ class DB( ServiceDB ):
if service.CanDownload():
serverside_message_keys = self.Read( 'message_keys_to_download', HC.LOW_PRIORITY, service_identifier )
serverside_message_keys = wx.GetApp().ReadDaemon( 'message_keys_to_download', service_identifier )
if len( serverside_message_keys ) > 0:
@ -6524,7 +6600,7 @@ class DB( ServiceDB ):
message = HydrusMessageHandling.UnpackageDeliveredMessage( encrypted_message, private_key )
self.Write( 'message', HC.LOW_PRIORITY, message, serverside_message_key = serverside_message_key )
wx.GetApp().WriteDaemon( 'message', message, serverside_message_key = serverside_message_key )
num_processed += 1
@ -6551,15 +6627,15 @@ class DB( ServiceDB ):
self.Write( 'flush_message_statuses', HC.LOW_PRIORITY )
wx.GetApp().WriteDaemon( 'flush_message_statuses' )
# send messages to recipients and update my status to sent/failed
messages_to_send = self.Read( 'messages_to_send', HC.LOW_PRIORITY )
messages_to_send = wx.GetApp().ReadDaemon( 'messages_to_send' )
for ( message_key, contacts_to ) in messages_to_send:
message = self.Read( 'transport_message', HC.LOW_PRIORITY, message_key )
message = wx.GetApp().ReadDaemon( 'transport_message', message_key )
contact_from = message.GetContactFrom()
@ -6570,7 +6646,7 @@ class DB( ServiceDB ):
my_public_key = contact_from.GetPublicKey()
my_contact_key = contact_from.GetContactKey()
my_message_depot = self.Read( 'service', HC.LOW_PRIORITY, contact_from )
my_message_depot = wx.GetApp().ReadDaemon( 'service', contact_from )
from_connection = my_message_depot.GetConnection()
@ -6611,19 +6687,19 @@ class DB( ServiceDB ):
if not from_anon: from_connection.Post( 'message_statuses', contact_key = my_contact_key, statuses = service_status_updates )
self.Write( 'message_statuses', HC.LOW_PRIORITY, message_key, local_status_updates )
wx.GetApp().WriteDaemon( 'message_statuses', message_key, local_status_updates )
self.Read( 'status_num_inbox', HC.LOW_PRIORITY )
wx.GetApp().ReadDaemon( 'status_num_inbox' )
def DAEMONSynchroniseRepositoriesAndSubscriptions( self ):
# repos
HC.repos_or_subs_changed = False
if not self._options[ 'pause_repo_sync' ]:
service_identifiers = self.Read( 'service_identifiers', HC.LOW_PRIORITY, HC.REPOSITORIES )
service_identifiers = wx.GetApp().ReadDaemon( 'service_identifiers', HC.REPOSITORIES )
for service_identifier in service_identifiers:
@ -6635,7 +6711,7 @@ class DB( ServiceDB ):
service_type = service_identifier.GetType()
try: service = self.Read( 'service', HC.LOW_PRIORITY, service_identifier )
try: service = wx.GetApp().ReadDaemon( 'service', service_identifier )
except: continue
if service.CanUpdate():
@ -6644,8 +6720,6 @@ class DB( ServiceDB ):
while service.CanUpdate():
paused_i = 0
while self._options[ 'pause_repo_sync' ]:
HC.pubsub.pub( 'service_status', 'Repository synchronisation paused' )
@ -6654,9 +6728,12 @@ class DB( ServiceDB ):
if HC.shutdown: raise Exception( 'Application shutting down!' )
if paused_i == 10: return
paused_i += 1
if HC.repos_or_subs_changed:
HC.pubsub.pub( 'service_status', 'Sync daemon restarting' )
return
if HC.shutdown: raise Exception( 'Application shutting down!' )
@ -6676,7 +6753,7 @@ class DB( ServiceDB ):
HC.pubsub.pub( 'service_status', 'Generating tags for ' + name )
self.Write( 'generate_tag_ids', HC.LOW_PRIORITY, update.GetTags() )
wx.GetApp().WriteDaemon( 'generate_tag_ids', update.GetTags() )
updates = update.SplitIntoSubUpdates()
@ -6685,7 +6762,7 @@ class DB( ServiceDB ):
for ( i, sub_update ) in enumerate( updates ):
self.Write( 'update', HC.LOW_PRIORITY, service_identifier, sub_update )
wx.GetApp().WriteDaemon( 'update', service_identifier, sub_update )
HC.pubsub.pub( 'service_status', 'Processing ' + update_index_string + ' part ' + str( i + 1 ) + '/' + str( num_updates ) + ' for ' + name )
@ -6705,7 +6782,7 @@ class DB( ServiceDB ):
if now - timestamp < 86400 * 7: HC.pubsub.pub( 'message', service_identifier.GetName() + ' at ' + time.ctime( timestamp ) + ':' + os.linesep + os.linesep + news )
try: service = self.Read( 'service', HC.LOW_PRIORITY, service_identifier )
try: service = wx.GetApp().ReadDaemon( 'service', service_identifier )
except: break
@ -6734,9 +6811,7 @@ class DB( ServiceDB ):
if not self._options[ 'pause_subs_sync' ]:
subscriptions = wx.GetApp().Read( 'subscriptions' )
updated_subscriptions = []
subscriptions = wx.GetApp().ReadDaemon( 'subscriptions' )
for ( site_download_type, name, query_type, query, frequency_type, frequency_number, advanced_tag_options, advanced_import_options, last_checked, url_cache ) in subscriptions:
@ -6758,7 +6833,7 @@ class DB( ServiceDB ):
if site_download_type == HC.SITE_DOWNLOAD_TYPE_BOORU:
try: booru = wx.GetApp().Read( 'booru', booru_name )
try: booru = wx.GetApp().ReadDaemon( 'booru', booru_name )
except: raise Exception( 'While attempting to execute a subscription on booru ' + name + ', the client could not find that booru in the db.' )
tags = query.split( ' ' )
@ -6814,8 +6889,6 @@ class DB( ServiceDB ):
while True:
paused_i = 0
while self._options[ 'pause_subs_sync' ]:
HC.pubsub.pub( 'service_status', 'Subscription synchronisation paused' )
@ -6824,9 +6897,12 @@ class DB( ServiceDB ):
if HC.shutdown: return
if paused_i == 10: return
paused_i += 1
if HC.repos_or_subs_changed:
HC.pubsub.pub( 'service_status', 'Sync daemon restarting' )
return
if HC.shutdown: return
@ -6864,8 +6940,6 @@ class DB( ServiceDB ):
for url_args in all_url_args:
paused_i = 0
while self._options[ 'pause_subs_sync' ]:
HC.pubsub.pub( 'service_status', 'Subscription synchronisation paused' )
@ -6874,9 +6948,12 @@ class DB( ServiceDB ):
if HC.shutdown: return
if paused_i == 10: return
paused_i += 1
if HC.repos_or_subs_changed:
HC.pubsub.pub( 'service_status', 'Sync daemon restarting' )
return
if HC.shutdown: return
@ -6889,7 +6966,7 @@ class DB( ServiceDB ):
HC.pubsub.pub( 'service_status', name + ': ' + x_out_of_y + ' : checking url status' )
( status, hash ) = wx.GetApp().Read( 'url_status', url )
( status, hash ) = wx.GetApp().ReadDaemon( 'url_status', url )
if status == 'deleted' and 'exclude_deleted_files' not in advanced_import_options: status = 'new'
@ -6956,11 +7033,13 @@ class DB( ServiceDB ):
if site_download_type == HC.SITE_DOWNLOAD_TYPE_BOORU: query_type = ( booru_name, query_type )
updated_subscriptions.append( ( site_download_type, name, query_type, query, frequency_type, frequency_number, advanced_tag_options, advanced_import_options, last_checked, url_cache ) )
subscription = ( site_download_type, name, query_type, query, frequency_type, frequency_number, advanced_tag_options, advanced_import_options, last_checked, url_cache )
wx.GetApp().Write( 'subscription', subscription )
wx.GetApp().Write( 'subscriptions', updated_subscriptions )
HC.pubsub.pub( 'service_status', '' )
def ProcessRequest( self, request_type, request, request_args ):
@ -6973,7 +7052,7 @@ class DB( ServiceDB ):
hash = request_args[ 'hash' ]
file = self.Read( 'file', HC.HIGH_PRIORITY, hash )
file = wx.GetApp().ReadDaemon( 'file', hash )
mime = HC.GetMimeFromString( file )
@ -6983,7 +7062,7 @@ class DB( ServiceDB ):
hash = request_args[ 'hash' ]
thumbnail = self.Read( 'thumbnail', HC.HIGH_PRIORITY, hash )
thumbnail = wx.GetApp().ReadDaemon( 'thumbnail', hash )
mime = HC.GetMimeFromString( thumbnail )
@ -7149,6 +7228,7 @@ class DB( ServiceDB ):
elif action == 'session': self._AddSession( c, *args, **kwargs )
elif action == 'set_password': self._SetPassword( c, *args, **kwargs )
elif action == 'set_tag_service_precedence': self._SetTagServicePrecedence( c, *args, **kwargs )
elif action == 'subscription': self._SetSubscription( c, *args, **kwargs )
elif action == 'subscriptions': self._SetSubscriptions( c, *args, **kwargs )
elif action == 'thumbnails': self._AddThumbnails( c, *args, **kwargs )
elif action == 'update': self._AddUpdate( c, *args, **kwargs )

18
include/ClientGUI.py
View File

@ -502,12 +502,18 @@ class FrameGUI( ClientGUICommon.Frame ):
def _EditServices( self ):
original_pause_status = self._options[ 'pause_repo_sync' ]
self._options[ 'pause_repo_sync' ] = True
try:
with ClientGUIDialogs.DialogManageServices( self ) as dlg: dlg.ShowModal()
except: wx.MessageBox( traceback.format_exc() )
self._options[ 'pause_repo_sync' ] = original_pause_status
def _FetchIP( self, service_identifier ):
@ -646,12 +652,18 @@ class FrameGUI( ClientGUICommon.Frame ):
def _ManageSubscriptions( self ):
original_pause_status = self._options[ 'pause_subs_sync' ]
self._options[ 'pause_subs_sync' ] = True
try:
with ClientGUIDialogs.DialogManageSubscriptions( self ) as dlg: dlg.ShowModal()
except Exception as e: wx.MessageBox( unicode( e ) + traceback.format_exc() )
self._options[ 'pause_subs_sync' ] = original_pause_status
def _ManageTagServicePrecedence( self ):
@ -1796,12 +1808,18 @@ class FrameReviewServices( ClientGUICommon.Frame ):
def EventEdit( self, event ):
original_pause_status = self._options[ 'pause_repo_sync' ]
self._options[ 'pause_repo_sync' ] = True
try:
with ClientGUIDialogs.DialogManageServices( self ) as dlg: dlg.ShowModal()
except: wx.MessageBox( traceback.format_exc() )
self._options[ 'pause_repo_sync' ] = original_pause_status
def EventOk( self, event ): self.Destroy()

10
include/ClientGUICanvas.py
View File

@ -419,7 +419,7 @@ class Canvas():
file_hash = self._current_display_media.GetHash()
self._media_window.movie = HC.CLIENT_FILES_DIR + os.path.sep + file_hash.encode( 'hex' ) + '.swf'
self._media_window.movie = CC.GetFilePath( file_hash, HC.APPLICATION_FLASH )
elif self._current_display_media.GetMime() == HC.VIDEO_FLV:
@ -428,7 +428,7 @@ class Canvas():
file_hash = self._current_display_media.GetHash()
flash_vars = []
flash_vars.append( ( 'flv', HC.CLIENT_FILES_DIR + os.path.sep + file_hash.encode( 'hex' ) + '.flv' ) )
flash_vars.append( ( 'flv', CC.GetFilePath( file_hash, HC.VIDEO_FLV ) ) )
flash_vars.append( ( 'margin', '0' ) )
flash_vars.append( ( 'autoload', '1' ) )
flash_vars.append( ( 'autoplay', '1' ) )
@ -980,7 +980,7 @@ class CanvasFullscreenMediaListBrowser( CanvasFullscreenMediaList ):
if wx.TheClipboard.Open():
data = wx.TextDataObject( HC.CLIENT_FILES_DIR + os.path.sep + self._current_media.GetHash().encode( 'hex' ) + HC.mime_ext_lookup[ self._current_media.GetMime() ] )
data = wx.TextDataObject( CC.GetFilePath( self._current_media.GetHash(), self._current_media.GetMime() ) )
wx.TheClipboard.SetData( data )
@ -1247,7 +1247,7 @@ class CanvasFullscreenMediaListCustomFilter( CanvasFullscreenMediaList ):
if wx.TheClipboard.Open():
data = wx.TextDataObject( HC.CLIENT_FILES_DIR + os.path.sep + self._current_media.GetHash().encode( 'hex' ) + HC.mime_ext_lookup[ self._current_media.GetMime() ] )
data = wx.TextDataObject( CC.GetFilePath( self._current_media.GetHash(), self._current_media.GetMime() ) )
wx.TheClipboard.SetData( data )
@ -2942,7 +2942,7 @@ class PDFButton( wx.Button ):
def EventButton( self, event ):
path = HC.CLIENT_FILES_DIR + os.path.sep + self._hash.encode( 'hex' ) + '.pdf'
path = CC.GetFilePath( self._hash, HC.APPLICATION_PDF )
# os.system( 'start ' + path )
subprocess.call( 'start "" "' + path + '"', shell = True )

131
include/ClientGUICommon.py
View File

@ -1955,6 +1955,137 @@ class OnOffButton( wx.Button ):
def IsOn( self ): return self._on
class RegexButton( wx.Button ):
ID_REGEX_WHITESPACE = 0
ID_REGEX_NUMBER = 1
ID_REGEX_ALPHANUMERIC = 2
ID_REGEX_ANY = 3
ID_REGEX_BEGINNING = 4
ID_REGEX_END = 5
ID_REGEX_0_OR_MORE_GREEDY = 6
ID_REGEX_1_OR_MORE_GREEDY = 7
ID_REGEX_0_OR_1_GREEDY = 8
ID_REGEX_0_OR_MORE_MINIMAL = 9
ID_REGEX_1_OR_MORE_MINIMAL = 10
ID_REGEX_0_OR_1_MINIMAL = 11
ID_REGEX_EXACTLY_M = 12
ID_REGEX_M_TO_N_GREEDY = 13
ID_REGEX_M_TO_N_MINIMAL = 14
ID_REGEX_LOOKAHEAD = 15
ID_REGEX_NEGATIVE_LOOKAHEAD = 16
ID_REGEX_LOOKBEHIND = 17
ID_REGEX_NEGATIVE_LOOKBEHIND = 18
ID_REGEX_NUMBER_WITHOUT_ZEROES = 19
ID_REGEX_NUMBER_EXT = 20
ID_REGEX_AUTHOR = 21
ID_REGEX_BACKSPACE = 22
ID_REGEX_SET = 23
ID_REGEX_NOT_SET = 24
def __init__( self, parent ):
wx.Button.__init__( self, parent, label = 'regex shortcuts' )
self.Bind( wx.EVT_BUTTON, self.EventButton )
self.Bind( wx.EVT_MENU, self.EventMenu )
def EventButton( self, event ):
menu = wx.Menu()
menu.Append( -1, 'click on a phrase to copy to clipboard' )
menu.AppendSeparator()
menu.Append( self.ID_REGEX_WHITESPACE, r'whitespace character - \s' )
menu.Append( self.ID_REGEX_NUMBER, r'number character - \d' )
menu.Append( self.ID_REGEX_ALPHANUMERIC, r'alphanumeric or backspace character - \w' )
menu.Append( self.ID_REGEX_ANY, r'any character - .' )
menu.Append( self.ID_REGEX_BACKSPACE, r'backspace character - \\' )
menu.Append( self.ID_REGEX_BEGINNING, r'beginning of line - ^' )
menu.Append( self.ID_REGEX_END, r'end of line - $' )
menu.Append( self.ID_REGEX_SET, r'any of these - [...]' )
menu.Append( self.ID_REGEX_NOT_SET, r'anything other than these - [^...]' )
menu.AppendSeparator()
menu.Append( self.ID_REGEX_0_OR_MORE_GREEDY, r'0 or more matches, consuming as many as possible - *' )
menu.Append( self.ID_REGEX_1_OR_MORE_GREEDY, r'1 or more matches, consuming as many as possible - +' )
menu.Append( self.ID_REGEX_0_OR_1_GREEDY, r'0 or 1 matches, preferring 1 - ?' )
menu.Append( self.ID_REGEX_0_OR_MORE_MINIMAL, r'0 or more matches, consuming as few as possible - *?' )
menu.Append( self.ID_REGEX_1_OR_MORE_MINIMAL, r'1 or more matches, consuming as few as possible - +?' )
menu.Append( self.ID_REGEX_0_OR_1_MINIMAL, r'0 or 1 matches, preferring 0 - *' )
menu.Append( self.ID_REGEX_EXACTLY_M, r'exactly m matches - {m}' )
menu.Append( self.ID_REGEX_M_TO_N_GREEDY, r'm to n matches, consuming as many as possible - {m,n}' )
menu.Append( self.ID_REGEX_M_TO_N_MINIMAL, r'm to n matches, consuming as few as possible - {m,n}?' )
menu.AppendSeparator()
menu.Append( self.ID_REGEX_LOOKAHEAD, r'the next characters are: (non-consuming) - (?=...)' )
menu.Append( self.ID_REGEX_NEGATIVE_LOOKAHEAD, r'the next characters are not: (non-consuming) - (?!...)' )
menu.Append( self.ID_REGEX_LOOKBEHIND, r'the previous characters are: (non-consuming) - (?<=...)' )
menu.Append( self.ID_REGEX_NEGATIVE_LOOKBEHIND, r'the previous characters are not: (non-consuming) - (?<!...)' )
menu.AppendSeparator()
menu.Append( self.ID_REGEX_NUMBER_WITHOUT_ZEROES, r'0074 -> 74 - [1-9]+\d*' )
menu.Append( self.ID_REGEX_NUMBER_EXT, r'...0074.jpg -> 74 - [1-9]+\d*(?=.{4}$)' )
menu.Append( self.ID_REGEX_AUTHOR, r'E:\my collection\author name - v4c1p0074.jpg -> author name - [^\\][\w\s]*(?=\s-)' )
self.PopupMenu( menu )
menu.Destroy()
def EventMenu( self, event ):
id = event.GetId()
phrase = None
if id == self.ID_REGEX_WHITESPACE: phrase = r'\s'
elif id == self.ID_REGEX_NUMBER: phrase = r'\d'
elif id == self.ID_REGEX_ALPHANUMERIC: phrase = r'\w'
elif id == self.ID_REGEX_ANY: phrase = r'.'
elif id == self.ID_REGEX_BACKSPACE: phrase = r'\\'
elif id == self.ID_REGEX_BEGINNING: phrase = r'^'
elif id == self.ID_REGEX_END: phrase = r'$'
elif id == self.ID_REGEX_SET: phrase = r'[...]'
elif id == self.ID_REGEX_NOT_SET: phrase = r'[^...]'
elif id == self.ID_REGEX_0_OR_MORE_GREEDY: phrase = r'*'
elif id == self.ID_REGEX_1_OR_MORE_GREEDY: phrase = r'+'
elif id == self.ID_REGEX_0_OR_1_GREEDY: phrase = r'?'
elif id == self.ID_REGEX_0_OR_MORE_MINIMAL: phrase = r'*?'
elif id == self.ID_REGEX_1_OR_MORE_MINIMAL: phrase = r'+?'
elif id == self.ID_REGEX_0_OR_1_MINIMAL: phrase = r'*'
elif id == self.ID_REGEX_EXACTLY_M: phrase = r'{m}'
elif id == self.ID_REGEX_M_TO_N_GREEDY: phrase = r'{m,n}'
elif id == self.ID_REGEX_M_TO_N_MINIMAL: phrase = r'{m,n}?'
elif id == self.ID_REGEX_LOOKAHEAD: phrase = r'(?=...)'
elif id == self.ID_REGEX_NEGATIVE_LOOKAHEAD: phrase = r'(?!...)'
elif id == self.ID_REGEX_LOOKBEHIND: phrase = r'(?<=...)'
elif id == self.ID_REGEX_NEGATIVE_LOOKBEHIND: phrase = r'(?<!...)'
elif id == self.ID_REGEX_NUMBER_WITHOUT_ZEROES: phrase = r'[1-9]+\d*'
elif id == self.ID_REGEX_NUMBER_EXT: phrase = r'[1-9]+\d*(?=.{4}$)'
elif id == self.ID_REGEX_AUTHOR: phrase = r'[^\\][\w\s]*(?=\s-)'
else: event.Skip()
if phrase is not None:
if wx.TheClipboard.Open():
data = wx.TextDataObject( phrase )
wx.TheClipboard.SetData( data )
wx.TheClipboard.Close()
else: wx.MessageBox( 'I could not get permission to access the clipboard.' )
class SaneListCtrl( wx.ListCtrl, ListCtrlAutoWidthMixin, ColumnSorterMixin ):
def __init__( self, parent, height, columns ):

303
include/ClientGUIDialogs.py
View File

@ -662,6 +662,74 @@ class DialogInputCustomFilterAction( Dialog ):
def SetTag( self, tag ): self._tag_value.SetValue( tag )
class DialogInputNamespaceRegex( Dialog ):
def __init__( self, parent, namespace = '', regex = '' ):
def InitialiseControls():
self._namespace = wx.TextCtrl( self )
self._namespace.SetValue( namespace )
self._regex = wx.TextCtrl( self )
self._regex.SetValue( regex )
self._shortcuts = ClientGUICommon.RegexButton( self )
self._ok = wx.Button( self, label='Ok' )
self._ok.Bind( wx.EVT_BUTTON, self.EventOk )
self._ok.SetForegroundColour( ( 0, 128, 0 ) )
self._cancel = wx.Button( self, id = wx.ID_CANCEL, label='Cancel' )
self._cancel.Bind( wx.EVT_BUTTON, self.EventCancel )
self._cancel.SetForegroundColour( ( 128, 0, 0 ) )
def InitialisePanel():
control_box = wx.BoxSizer( wx.HORIZONTAL )
control_box.AddF( self._namespace, FLAGS_EXPAND_BOTH_WAYS )
control_box.AddF( wx.StaticText( self, label = ':' ), FLAGS_MIXED )
control_box.AddF( self._regex, FLAGS_EXPAND_BOTH_WAYS )
b_box = wx.BoxSizer( wx.HORIZONTAL )
b_box.AddF( self._ok, FLAGS_MIXED )
b_box.AddF( self._cancel, FLAGS_MIXED )
vbox = wx.BoxSizer( wx.VERTICAL )
vbox.AddF( control_box, FLAGS_EXPAND_SIZER_PERPENDICULAR )
vbox.AddF( self._shortcuts, FLAGS_LONE_BUTTON )
vbox.AddF( b_box, FLAGS_BUTTON_SIZERS )
self.SetSizer( vbox )
( x, y ) = self.GetEffectiveMinSize()
self.SetInitialSize( ( x, y ) )
Dialog.__init__( self, parent, 'configure quick namespace' )
InitialiseControls()
InitialisePanel()
def EventCancel( self, event ): self.EndModal( wx.ID_CANCEL )
def EventOk( self, event ): self.EndModal( wx.ID_OK )
def GetInfo( self ):
namespace = self._namespace.GetValue()
regex = self._regex.GetValue()
return ( namespace, regex )
class DialogInputNewAccounts( Dialog ):
def __init__( self, parent, service_identifier ):
@ -6555,9 +6623,9 @@ class DialogManageSubscriptions( Dialog ):
listbook = types_to_listbooks[ site_download_type ]
page_info = ( self._Panel, ( listbook, site_download_type, name, query_type, query, frequency_type, frequency_number, advanced_tag_options, advanced_import_options, last_checked, url_cache ), {} )
page = self._Panel( listbook, site_download_type, name, query_type, query, frequency_type, frequency_number, advanced_tag_options, advanced_import_options, last_checked, url_cache )
listbook.AddPage( page_info, name )
listbook.AddPage( page, name )
self._listbook.AddPage( self._deviant_art, 'deviant art' )
@ -6778,8 +6846,6 @@ class DialogManageSubscriptions( Dialog ):
try: wx.GetApp().Write( 'subscriptions', subscriptions )
except Exception as e: wx.MessageBox( 'Saving services to DB raised this error: ' + unicode( e ) )
HC.pubsub.pub( 'notify_new_subscriptions' )
self.EndModal( wx.ID_OK )
@ -7009,8 +7075,6 @@ class DialogManageSubscriptions( Dialog ):
for ( title, timespan ) in ( ( 'days', 86400 ), ( 'weeks', 86400 * 7 ), ( 'months', 86400 * 30 ) ):
self._frequency_type.Append( title, timespan )
if frequency_type == timespan: index_to_select = i
i += 1
@ -8071,37 +8135,11 @@ class DialogPathsToTagsRegex( Dialog ):
class _Panel( wx.Panel ):
ID_REGEX_WHITESPACE = 0
ID_REGEX_NUMBER = 1
ID_REGEX_ALPHANUMERIC = 2
ID_REGEX_ANY = 3
ID_REGEX_BEGINNING = 4
ID_REGEX_END = 5
ID_REGEX_0_OR_MORE_GREEDY = 6
ID_REGEX_1_OR_MORE_GREEDY = 7
ID_REGEX_0_OR_1_GREEDY = 8
ID_REGEX_0_OR_MORE_MINIMAL = 9
ID_REGEX_1_OR_MORE_MINIMAL = 10
ID_REGEX_0_OR_1_MINIMAL = 11
ID_REGEX_EXACTLY_M = 12
ID_REGEX_M_TO_N_GREEDY = 13
ID_REGEX_M_TO_N_MINIMAL = 14
ID_REGEX_LOOKAHEAD = 15
ID_REGEX_NEGATIVE_LOOKAHEAD = 16
ID_REGEX_LOOKBEHIND = 17
ID_REGEX_NEGATIVE_LOOKBEHIND = 18
ID_REGEX_NUMBER_WITHOUT_ZEROES = 19
ID_REGEX_NUMBER_EXT = 20
ID_REGEX_AUTHOR = 21
ID_REGEX_BACKSPACE = 22
ID_REGEX_SET = 23
ID_REGEX_NOT_SET = 24
def __init__( self, parent, service_identifier, paths ):
def InitialiseControls():
self._paths_list = ClientGUICommon.SaneListCtrl( self, 300, [ ( 'path', 400 ), ( 'tags', -1 ) ] )
self._paths_list = ClientGUICommon.SaneListCtrl( self, 250, [ ( 'path', 400 ), ( 'tags', -1 ) ] )
self._paths_list.Bind( wx.EVT_LIST_ITEM_SELECTED, self.EventItemSelected )
self._paths_list.Bind( wx.EVT_LIST_ITEM_DESELECTED, self.EventItemSelected )
@ -8110,18 +8148,18 @@ class DialogPathsToTagsRegex( Dialog ):
self._quick_namespaces_panel = ClientGUICommon.StaticBox( self, 'quick namespaces' )
self._page_regex = wx.TextCtrl( self._quick_namespaces_panel )
self._chapter_regex = wx.TextCtrl( self._quick_namespaces_panel )
self._volume_regex = wx.TextCtrl( self._quick_namespaces_panel )
self._title_regex = wx.TextCtrl( self._quick_namespaces_panel )
self._series_regex = wx.TextCtrl( self._quick_namespaces_panel )
self._creator_regex = wx.TextCtrl( self._quick_namespaces_panel )
self._quick_namespaces_list = ClientGUICommon.SaneListCtrl( self._quick_namespaces_panel, 200, [ ( 'namespace', 80 ), ( 'regex', -1 ) ] )
self._update_button = wx.Button( self._quick_namespaces_panel, label='update' )
self._update_button.Bind( wx.EVT_BUTTON, self.EventUpdate )
self._add_quick_namespace_button = wx.Button( self._quick_namespaces_panel, label = 'add' )
self._add_quick_namespace_button.Bind( wx.EVT_BUTTON, self.EventAddQuickNamespace )
self._regex_shortcuts = wx.Button( self._quick_namespaces_panel, label = 'regex shortcuts' )
self._regex_shortcuts.Bind( wx.EVT_BUTTON, self.EventRegexShortcuts )
self._edit_quick_namespace_button = wx.Button( self._quick_namespaces_panel, label = 'edit' )
self._edit_quick_namespace_button.Bind( wx.EVT_BUTTON, self.EventEditQuickNamespace )
self._delete_quick_namespace_button = wx.Button( self._quick_namespaces_panel, label = 'delete' )
self._delete_quick_namespace_button.Bind( wx.EVT_BUTTON, self.EventDeleteQuickNamespace )
self._regex_shortcuts = ClientGUICommon.RegexButton( self._quick_namespaces_panel )
self._regex_link = wx.HyperlinkCtrl( self._quick_namespaces_panel, id = -1, label = 'a good regex introduction', url = 'http://www.aivosto.com/vbtips/regex.html' )
@ -8166,25 +8204,14 @@ class DialogPathsToTagsRegex( Dialog ):
def InitialisePanel():
gridbox = wx.FlexGridSizer( 0, 2 )
button_box = wx.BoxSizer( wx.HORIZONTAL )
gridbox.AddGrowableCol( 1, 1 )
button_box.AddF( self._add_quick_namespace_button, FLAGS_MIXED )
button_box.AddF( self._edit_quick_namespace_button, FLAGS_MIXED )
button_box.AddF( self._delete_quick_namespace_button, FLAGS_MIXED )
gridbox.AddF( wx.StaticText( self._quick_namespaces_panel, label='Page regex ' ), FLAGS_MIXED )
gridbox.AddF( self._page_regex, FLAGS_EXPAND_BOTH_WAYS )
gridbox.AddF( wx.StaticText( self._quick_namespaces_panel, label='Chapter regex ' ), FLAGS_MIXED )
gridbox.AddF( self._chapter_regex, FLAGS_EXPAND_BOTH_WAYS )
gridbox.AddF( wx.StaticText( self._quick_namespaces_panel, label='Volume regex ' ), FLAGS_MIXED )
gridbox.AddF( self._volume_regex, FLAGS_EXPAND_BOTH_WAYS )
gridbox.AddF( wx.StaticText( self._quick_namespaces_panel, label='Title regex ' ), FLAGS_MIXED )
gridbox.AddF( self._title_regex, FLAGS_EXPAND_BOTH_WAYS )
gridbox.AddF( wx.StaticText( self._quick_namespaces_panel, label='Series regex ' ), FLAGS_MIXED )
gridbox.AddF( self._series_regex, FLAGS_EXPAND_BOTH_WAYS )
gridbox.AddF( wx.StaticText( self._quick_namespaces_panel, label='Creator regex ' ), FLAGS_MIXED )
gridbox.AddF( self._creator_regex, FLAGS_EXPAND_BOTH_WAYS )
self._quick_namespaces_panel.AddF( gridbox, FLAGS_EXPAND_SIZER_PERPENDICULAR )
self._quick_namespaces_panel.AddF( self._update_button, FLAGS_LONE_BUTTON )
self._quick_namespaces_panel.AddF( self._quick_namespaces_list, FLAGS_EXPAND_PERPENDICULAR )
self._quick_namespaces_panel.AddF( button_box, FLAGS_BUTTON_SIZERS )
self._quick_namespaces_panel.AddF( self._regex_shortcuts, FLAGS_LONE_BUTTON )
self._quick_namespaces_panel.AddF( self._regex_link, FLAGS_LONE_BUTTON )
@ -8221,8 +8248,6 @@ class DialogPathsToTagsRegex( Dialog ):
InitialisePanel()
self.Bind( wx.EVT_MENU, self.EventMenu )
def _GetTags( self, path ):
@ -8247,26 +8272,17 @@ class DialogPathsToTagsRegex( Dialog ):
except: pass
namespaced_regexes = []
namespaced_regexes.append( ( self._page_regex, 'page:' ) )
namespaced_regexes.append( ( self._chapter_regex, 'chapter:' ) )
namespaced_regexes.append( ( self._volume_regex, 'volume:' ) )
namespaced_regexes.append( ( self._title_regex, 'title:' ) )
namespaced_regexes.append( ( self._series_regex, 'series:' ) )
namespaced_regexes.append( ( self._creator_regex, 'creator:' ) )
for ( control, prefix ) in namespaced_regexes:
for ( namespace, regex ) in self._quick_namespaces_list.GetClientData():
try:
m = re.search( control.GetValue(), path )
m = re.search( regex, path )
if m is not None:
match = m.group()
if len( match ) > 0: tags.append( prefix + match )
if len( match ) > 0: tags.append( namespace + ':' + match )
except: pass
@ -8343,6 +8359,48 @@ class DialogPathsToTagsRegex( Dialog ):
def EventAddQuickNamespace( self, event ):
with DialogInputNamespaceRegex( self ) as dlg:
if dlg.ShowModal() == wx.ID_OK:
( namespace, regex ) = dlg.GetInfo()
self._quick_namespaces_list.Append( ( namespace, regex ), ( namespace, regex ) )
self._RefreshFileList()
def EventDeleteQuickNamespace( self, event ):
self._quick_namespaces_list.RemoveAllSelected()
self._RefreshFileList()
def EventEditQuickNamespace( self, event ):
for index in self._quick_namespaces_list.GetAllSelected():
( namespace, regex ) = self._quick_namespaces_list.GetClientData( index = index )
with DialogInputNamespaceRegex( self, namespace = namespace, regex = regex ) as dlg:
if dlg.ShowModal() == wx.ID_OK:
( namespace, regex ) = dlg.GetInfo()
self._quick_namespaces_list.UpdateRow( index, ( namespace, regex ), ( namespace, regex ) )
self._RefreshFileList()
def EventItemSelected( self, event ):
single_tags = set()
@ -8369,101 +8427,6 @@ class DialogPathsToTagsRegex( Dialog ):
self._single_tags.SetTags( single_tags )
def EventMenu( self, event ):
id = event.GetId()
phrase = None
if id == self.ID_REGEX_WHITESPACE: phrase = r'\s'
elif id == self.ID_REGEX_NUMBER: phrase = r'\d'
elif id == self.ID_REGEX_ALPHANUMERIC: phrase = r'\w'
elif id == self.ID_REGEX_ANY: phrase = r'.'
elif id == self.ID_REGEX_BACKSPACE: phrase = r'\\'
elif id == self.ID_REGEX_BEGINNING: phrase = r'^'
elif id == self.ID_REGEX_END: phrase = r'$'
elif id == self.ID_REGEX_SET: phrase = r'[...]'
elif id == self.ID_REGEX_NOT_SET: phrase = r'[^...]'
elif id == self.ID_REGEX_0_OR_MORE_GREEDY: phrase = r'*'
elif id == self.ID_REGEX_1_OR_MORE_GREEDY: phrase = r'+'
elif id == self.ID_REGEX_0_OR_1_GREEDY: phrase = r'?'
elif id == self.ID_REGEX_0_OR_MORE_MINIMAL: phrase = r'*?'
elif id == self.ID_REGEX_1_OR_MORE_MINIMAL: phrase = r'+?'
elif id == self.ID_REGEX_0_OR_1_MINIMAL: phrase = r'*'
elif id == self.ID_REGEX_EXACTLY_M: phrase = r'{m}'
elif id == self.ID_REGEX_M_TO_N_GREEDY: phrase = r'{m,n}'
elif id == self.ID_REGEX_M_TO_N_MINIMAL: phrase = r'{m,n}?'
elif id == self.ID_REGEX_LOOKAHEAD: phrase = r'(?=...)'
elif id == self.ID_REGEX_NEGATIVE_LOOKAHEAD: phrase = r'(?!...)'
elif id == self.ID_REGEX_LOOKBEHIND: phrase = r'(?<=...)'
elif id == self.ID_REGEX_NEGATIVE_LOOKBEHIND: phrase = r'(?<!...)'
elif id == self.ID_REGEX_NUMBER_WITHOUT_ZEROES: phrase = r'[1-9]+\d*'
elif id == self.ID_REGEX_NUMBER_EXT: phrase = r'[1-9]+\d*(?=.{4}$)'
elif id == self.ID_REGEX_AUTHOR: phrase = r'[^\\][\w\s]*(?=\s-)'
else: event.Skip()
if phrase is not None:
if wx.TheClipboard.Open():
data = wx.TextDataObject( phrase )
wx.TheClipboard.SetData( data )
wx.TheClipboard.Close()
else: wx.MessageBox( 'I could not get permission to access the clipboard.' )
def EventRegexShortcuts( self, event ):
menu = wx.Menu()
menu.Append( -1, 'click on a phrase to copy to clipboard' )
menu.AppendSeparator()
menu.Append( self.ID_REGEX_WHITESPACE, r'whitespace character - \s' )
menu.Append( self.ID_REGEX_NUMBER, r'number character - \d' )
menu.Append( self.ID_REGEX_ALPHANUMERIC, r'alphanumeric or backspace character - \w' )
menu.Append( self.ID_REGEX_ANY, r'any character - .' )
menu.Append( self.ID_REGEX_BACKSPACE, r'backspace character - \\' )
menu.Append( self.ID_REGEX_BEGINNING, r'beginning of line - ^' )
menu.Append( self.ID_REGEX_END, r'end of line - $' )
menu.Append( self.ID_REGEX_SET, r'any of these - [...]' )
menu.Append( self.ID_REGEX_NOT_SET, r'anything other than these - [^...]' )
menu.AppendSeparator()
menu.Append( self.ID_REGEX_0_OR_MORE_GREEDY, r'0 or more matches, consuming as many as possible - *' )
menu.Append( self.ID_REGEX_1_OR_MORE_GREEDY, r'1 or more matches, consuming as many as possible - +' )
menu.Append( self.ID_REGEX_0_OR_1_GREEDY, r'0 or 1 matches, preferring 1 - ?' )
menu.Append( self.ID_REGEX_0_OR_MORE_MINIMAL, r'0 or more matches, consuming as few as possible - *?' )
menu.Append( self.ID_REGEX_1_OR_MORE_MINIMAL, r'1 or more matches, consuming as few as possible - +?' )
menu.Append( self.ID_REGEX_0_OR_1_MINIMAL, r'0 or 1 matches, preferring 0 - *' )
menu.Append( self.ID_REGEX_EXACTLY_M, r'exactly m matches - {m}' )
menu.Append( self.ID_REGEX_M_TO_N_GREEDY, r'm to n matches, consuming as many as possible - {m,n}' )
menu.Append( self.ID_REGEX_M_TO_N_MINIMAL, r'm to n matches, consuming as few as possible - {m,n}?' )
menu.AppendSeparator()
menu.Append( self.ID_REGEX_LOOKAHEAD, r'the next characters are: (non-consuming) - (?=...)' )
menu.Append( self.ID_REGEX_NEGATIVE_LOOKAHEAD, r'the next characters are not: (non-consuming) - (?!...)' )
menu.Append( self.ID_REGEX_LOOKBEHIND, r'the previous characters are: (non-consuming) - (?<=...)' )
menu.Append( self.ID_REGEX_NEGATIVE_LOOKBEHIND, r'the previous characters are not: (non-consuming) - (?<!...)' )
menu.AppendSeparator()
menu.Append( self.ID_REGEX_NUMBER_WITHOUT_ZEROES, r'0074 -> 74 - [1-9]+\d*' )
menu.Append( self.ID_REGEX_NUMBER_EXT, r'...0074.jpg -> 74 - [1-9]+\d*(?=.{4}$)' )
menu.Append( self.ID_REGEX_AUTHOR, r'E:\my collection\author name - v4c1p0074.jpg -> author name - [^\\][\w\s]*(?=\s-)' )
self.PopupMenu( menu )
menu.Destroy()
def EventRemoveRegex( self, event ):
selection = self._regexes.GetSelection()
@ -8478,8 +8441,6 @@ class DialogPathsToTagsRegex( Dialog ):
def EventUpdate( self, event ): self._RefreshFileList()
def GetServiceIdentifier( self ): return self._service_identifier
# this prob needs to be made cleverer if I do the extra column

6
include/ClientGUIManagement.py
View File

@ -1168,7 +1168,7 @@ class ManagementPanelImport( ManagementPanel ):
self._import_current_info.SetLabel( self._GetPreimportStatus() )
wx.GetApp().WriteLowPriority( 'import_file_from_page', self._page_key, file, advanced_import_options = advanced_import_options, service_identifiers_to_tags = service_identifiers_to_tags, url = url )
wx.GetApp().WriteDaemon( 'import_file_from_page', self._page_key, file, advanced_import_options = advanced_import_options, service_identifiers_to_tags = service_identifiers_to_tags, url = url )
else:
@ -1644,7 +1644,7 @@ class ManagementPanelImportWithQueueAdvanced( ManagementPanelImportWithQueue ):
content_updates = HydrusDownloading.ConvertServiceIdentifiersToTagsToContentUpdates( hash, service_identifiers_to_tags )
wx.GetApp().Write( 'content_updates', content_updates )
wx.GetApp().WriteDaemon( 'content_updates', content_updates )
HC.pubsub.pub( 'import_done', self._page_key, 'redundant' )
@ -2438,7 +2438,7 @@ class ManagementPanelPetitions( ManagementPanel ):
content_updates = [ HC.ContentUpdate( HC.CONTENT_UPDATE_DELETE, self._file_service_identifier, hashes, tag ) ]
wx.GetApp().Write( 'content_updates', content_updates )
wx.GetApp().WriteDaemon( 'content_updates', content_updates )
self._current_petition = None

2
include/ClientGUIMedia.py
View File

@ -145,7 +145,7 @@ class MediaPanel( ClientGUIMixins.ListeningMediaList, wx.ScrolledWindow ):
display_media = self._focussed_media.GetDisplayMedia()
data = wx.TextDataObject( HC.CLIENT_FILES_DIR + os.path.sep + display_media.GetHash().encode( 'hex' ) + HC.mime_ext_lookup[ display_media.GetMime() ] )
data = wx.TextDataObject( CC.GetFilePath( display_media.GetHash(), display_media.GetMime() ) )
wx.TheClipboard.SetData( data )

12
include/ClientGUIPages.py
View File

@ -364,7 +364,13 @@ class PageQuery( PageWithMedia ):
def _InitMediaPanel( self ):
if len( self._initial_media_results ) == 0: self._media_panel = ClientGUIMedia.MediaPanelNoQuery( self, self._page_key, self._file_service_identifier )
else: self._media_panel = ClientGUIMedia.MediaPanelThumbnails( self, self._page_key, self._file_service_identifier, self._initial_predicates, self._initial_media_results )
else:
file_query_result = CC.FileQueryResult( self._file_service_identifier, self._initial_predicates, self._initial_media_results )
self._media_panel = ClientGUIMedia.MediaPanelThumbnails( self, self._page_key, self._file_service_identifier, self._initial_predicates, file_query_result )
class PageThreadDumper( PageWithMedia ):
@ -382,10 +388,12 @@ class PageThreadDumper( PageWithMedia ):
self._media_results = filter( self._imageboard.IsOkToPost, self._media_results )
self._file_query_result = CC.FileQueryResult( HC.LOCAL_FILE_SERVICE_IDENTIFIER, [], self._media_results )
PageWithMedia.__init__( self, parent, HC.LOCAL_FILE_SERVICE_IDENTIFIER )
def _InitManagementPanel( self ): self._management_panel = ClientGUIManagement.ManagementPanelDumper( self._search_preview_split, self, self._page_key, self._imageboard, self._media_results )
def _InitMediaPanel( self ): self._media_panel = ClientGUIMedia.MediaPanelThumbnails( self, self._page_key, HC.LOCAL_FILE_SERVICE_IDENTIFIER, [], self._media_results )
def _InitMediaPanel( self ): self._media_panel = ClientGUIMedia.MediaPanelThumbnails( self, self._page_key, HC.LOCAL_FILE_SERVICE_IDENTIFIER, [], self._file_query_result )

3
include/HydrusConstants.py
View File

@ -30,7 +30,7 @@ TEMP_DIR = BASE_DIR + os.path.sep + 'temp'
# Misc
NETWORK_VERSION = 9
SOFTWARE_VERSION = 66
SOFTWARE_VERSION = 67
UNSCALED_THUMBNAIL_DIMENSIONS = ( 200, 200 )
@ -42,6 +42,7 @@ expirations = [ ( 'one month', 31 * 86400 ), ( 'three months', 3 * 31 * 86400 ),
shutdown = False
is_first_start = False
repos_or_subs_changed = False
# Enums