RepoMirrors
/
hydrus
mirror of https://github.com/hydrusnetwork/hydrus.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Version 186

This commit is contained in:
Hydrus 2015-12-16 16:41:06 -06:00
parent 39711a06eb
commit 2c80089fc3
22 changed files with 965 additions and 172 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
client.pyw
View File

@ -59,11 +59,18 @@ try:
HydrusGlobals.shutdown_complete = True
if HydrusGlobals.restart:
HydrusData.RestartProcess()
except:
import traceback
HydrusData.Print( 'Critical error occured! Details written to crash.log!' )
print( 'Critical error occured! Details written to crash.log!' )
with open( 'crash.log', 'wb' ) as f: f.write( traceback.format_exc() )

2
help/advanced.html
View File

@ -21,7 +21,7 @@
<h3>exclude deleted files</h3>
<p>In the client's options is a checkbox to exclude deleted files. It recurs pretty much anywhere you can import, under 'import file options'. If you select this, any file you ever deleted will be excluded from all future remote searches and import operations. This can stop you from importing/downloading and filtering out the same bad files several times over. The default is off. You may wish to have it set one way most of the time, but switch it the other just for one specific import or search.</p>
<h3>inputting non-english lanuages</h3>
<p>If you typically use an IME to input Japanese or another non-english language, you may have encountered problems entering into the autocomplete tag entry control in that you need Up/Down/Enter to navigate the IME, but the autocomplete steals those key presses away to navigate the list of results. To fix this, press Shift+Tab to temporarily disable the autocomplete's key event capture. The autocomplete text box will change colour to let you know it has released its normal key capture. Use your IME to get the text you want, then hit Shift+Tab again to restore the autocomplete to normal behaviour.</p>
<p>If you typically use an IME to input Japanese or another non-english language, you may have encountered problems entering into the autocomplete tag entry control in that you need Up/Down/Enter to navigate the IME, but the autocomplete steals those key presses away to navigate the list of results. To fix this, press Insert to temporarily disable the autocomplete's key event capture. The autocomplete text box will change colour to let you know it has released its normal key capture. Use your IME to get the text you want, then hit Insert again to restore the autocomplete to normal behaviour.</p>
<h3>tag censorship</h3>
<p>If you do not like a particular tag or namespace, you can easily hide it with <i>services->manage tag censorship</i>:</p>
<p><img src="tag_censorship.png" /></p>

28
help/changelog.html
View File

@ -8,6 +8,34 @@
<div class="content">
<h3>changelog</h3>
<ul>
<li><h3>version 186</h3></li>
<ul>
<ul>
<li>Windows release is now 64-bit!</li>
<li>some libraries got updated for windows, to varying minor effects</li>
<li>BeautifulSoup updated to latest version for everything</li>
<li>fixed a warning from newer versions of BeautifulSoup by explicitly using lxml</li>
<li>updated the running_from_source help file to what is currently needed</li>
<li>ffmpeg.exe is also 64-bit</li>
<li>wrote a "reducing program lag" page for the help</li>
<li>added program restart to file menu, doesn't work for linux yet</li>
<li>improved restart code (it was used in restore database, and for some users would not close the parent process down properly)</li>
<li>in the media viewer, page up/down, arrow keys and mouse scroll no longer unhide the mouse</li>
<li>in the media viewer under Windows, dragging no longer unhides the mouse</li>
<li>the implicit delay in the downloader-importers is reduced, so redundant files should always stream in a super fast now</li>
<li>export files dialog will now auto-create missing destination directories</li>
<li>export files dialog will now work off the gui thread (to stop gui hangups on big jobs) and will report its export status through the export button, which will be disabled until the job is finished</li>
<li>changed the new tab/shift+tab shortcuts because they break panel control navigation (I want to make these editable soon):</li>
<li>'fetch autocomplete results now' shortcut is now Ctrl+Space (Raw Control for OS X, not command)</li>
<li>'IME mode toggle' is now Insert</li>
<li>cleaned up how control is handled cross-platform--in general, 'control' is always control for windows and linux, and 'control' is command for os x. this may be incorrect.</li>
<li>json dump errors now report more information about the specific object and its serialisable contents that are causing the problem</li>
<li>clipped sibling/parent reason entry dialog message when lots of pairs are added</li>
<li>fixed a bit of bad logic in sibling/parent dialog->content-update workflow</li>
<li>fixed the thumbnail canvas not refreshing on a canvas shrink caused by expansion of the management panel that doesn't change the thumb column count</li>
<li>fixed emergency boot error reporting for when HydrusData won't load</li>
<li>added some unicode conversion unit tests</li>
</ul>
<li><h3>version 185</h3></li>
<ul>
<li>right clicking on a tag/predicate in the 'selection tags' box or the 'search' list of active predicates provides new intelligent menu options to discard/require/permit/exclude the selected tags from the current search. try it out!</li>

1
help/index.html
View File

@ -27,6 +27,7 @@
<li><a href="getting_started_ratings.html">getting started with ratings</a></li>
<li><a href="getting_started_subscriptions.html">getting started with subscriptions</a></li>
<li><a href="getting_started_local_booru.html">getting started with the local booru</a></li>
<li><a href="reducing_lag.html">reducing program lag</a></li>
<li><a href="access_keys.html">access keys to my server</a></li>
<li><a href="tagging_schema.html">thoughts on a public tagging schema</a></li>
<li><a href="advanced.html">advanced usage - general</a></li>

594
help/reducing_lag.html Normal file
View File

@ -0,0 +1,594 @@
<html>
<head>
<title>reducing lag</title>
<link href="hydrus.ico" rel="shortcut icon" />
<link href="style.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div class="content">
<h3>hydrus is cpu and hdd hungry</h3>
<p>The hydrus client manages a lot of complicated data and gives you a lot of power over it. To add millions of files and tags to its database, and then to perform difficult searches over that information, it needs to use a lot of CPU time and hard drive time--sometimes in small laggy blips, and occasionally in big 100% CPU chunks. I don't put training wheels or limiters on the software either, so if you search for 300,000 files, the client will try to fetch that many.</p>
<p>In general, the client works best on snappy computers with low-latency hard drives where it does not have to constantly compete with other CPU- or HDD- heavy programs. Running hydrus on your games computer is no problem at all, but you should have it set to not start a big job while your CPU is otherwise busy so your games can run freely. Similarly, if you run two clients on the same computer, you should have them set to work at different times, because if they both try to process 500,000 tags at once on the same hard drive, they will each slow to a crawl.</p>
<p>Keeping your HDDs defragged is very important, and good practise for all your programs anyway. Make sure you know what this is and that you do it. I use PerfectDisk. O&O Defrag is also good.</p>
<h3>maintenance and processing</h3>
<p>I have attempted to offload most of the background maintenance of the client (which typically means repository processing and internal database defragging) to time when you are not using the client. This can either be 'idle time' or 'shutdown time'. The calculations for what these exactly mean are customisable in <i>file->options->maintenance and processing</i>.</p>
<p>If you run a quick computer, you likely don't have to change any of these options. Repositories will synchronise and the database will stay fairly optimal without you even noticing the work that is going on. This is especially true if you leave your client on all the time.</p>
<p>If you have an old, slower computer though, or if your hard drive is high latency for one reason or another (e.g. you use encryption), make sure these options are set for whatever is best for your situation. Turning off idle time completely is often helpful as some older computers are slow to even recognise--mid task--that you want to use the client again, or take too long to abandon a big task half way through. If you set your client to only do work on shutdown, then you can control exactly when that happens.</p>
<p>Keeping the database vacuumed is important, so if you remove it from the normal maintenance schedule, make sure you run it every now and then manually from <i>database->maintenance->vacuum</i>. It takes a few minutes to run, but it is great for cleaning up a database recently fragged by several million new rows of data.</p>
<h3>reducing search and general gui lag</h3>
<p>Searching for tags via the autocomplete dropdown and searching for files in general can sometimes take a very long time. It depends on many things. In general, the more predicates (tags and system:something) you have active for a search, the faster it will be. And the more specific the search domain (e.g. "local files" instead of "all known files" and "my tag repo" instead of "all known tags"), the faster it will be.</p>
<p>You can also look at <i>file->options->speed and memory</i>, again especially if you have a slow computer. Increasing the autocomplete thresholds is very often helpful. You can even force autocompletes to only fetch results when you manually ask for them.</p>
<p>Having lots of thumbnails open can slow many things down. If you get lag with 10,000 files open in your searches, try cutting it down to only 1,000 or so. Split your downloading binges and subscriptions into smaller, rarer chunks, and don't try to watch 1080p webms while five other things are going on.</p>
<h3>finally</h3>
<p>Lots of my code remains unoptimised for certain situations. My development environment is obviously specific to me and has only a few thousand images and a few million tags. As I write code, I am usually more concerned with getting it to work at all rather than getting it to work fast for every possible scenario. So, if something is running particularly slow for you, but your computer is otherwise working fine, let me know and I can almost always speed it up.</p>
<p>Let me know:</p>
<ul>
<li>The general steps to reproduce the problem (e.g. "Running system:numtags>4 is ridiculously slow on its own on 'all known tags'.")</li>
<li>Your operating system and its version (e.g. "Windows 8.1")</li>
<li>Your computer's general power (e.g. "A couple of years old. It runs most stuff ok.")</li>
<li>The type of hard drive you are running hydrus from. (e.g. "A 2TB 7200rpm drive that is 20% full. I regularly defrag it.")</li>
<li>Any <i>profiles</i> you have collected.</li>
</ul>
<p>A <i>profile</i> is a block of debug text that lets me know which parts of my code are running slow for you. Hydrus reports look like this:</p>
<pre>2015/12/15 14:41:14: Profiling write import_file
2015/12/15 14:41:14: Stats
825 function calls in 0.188 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.077 0.077 0.077 0.077 {dct}
1 0.031 0.031 0.032 0.032 function_base.py:492(average)
1 0.023 0.023 0.023 0.023 {method 'decode' of 'ImagingDecoder' objects}
1 0.015 0.015 0.015 0.015 {cvtColor}
28 0.011 0.000 0.011 0.000 {method 'execute' of 'sqlite3.Cursor' objects}
1 0.007 0.007 0.007 0.007 {resize}
1 0.004 0.004 0.004 0.004 {method 'resize' of 'ImagingCore' objects}
1 0.004 0.004 0.136 0.136 ClientImageHandling.py:66(GeneratePerceptualHash)
1 0.003 0.003 0.003 0.003 {method 'encode' of 'ImagingEncoder' objects}
4 0.001 0.000 0.001 0.000 {method 'update' of '_hashlib.HASH' objects}
1 0.001 0.001 0.001 0.001 {imread}
9 0.001 0.000 0.001 0.000 {open}
4 0.001 0.000 0.001 0.000 {wx._core_.PostEvent}
1 0.001 0.001 0.001 0.001 {method 'any' of 'numpy.generic' objects}
4 0.000 0.000 0.000 0.000 {wx._core_.PyEvent__SetSelf}
71 0.000 0.000 0.000 0.000 {method 'read' of 'file' objects}
9 0.000 0.000 0.000 0.000 {method 'executemany' of 'sqlite3.Cursor' objects}
4 0.000 0.000 0.000 0.000 {wx._core_.Event_SetEventType}
1 0.000 0.000 0.000 0.000 {method '__enter__' of 'thread.lock' objects}
4 0.000 0.000 0.002 0.001 _core.py:16750(CallAfter)
1 0.000 0.000 0.145 0.145 ClientDB.py:1147(_AddThumbnails)
4 0.000 0.000 0.001 0.000 _core.py:7583(__init__)
4 0.000 0.000 0.003 0.001 HydrusPubSub.py:129(pub)
4 0.000 0.000 0.000 0.000 {wx._core_.GetApp}
1 0.000 0.000 0.183 0.183 ClientDB.py:4055(_ImportFile)
3 0.000 0.000 0.000 0.000 {method 'reduce' of 'numpy.ufunc' objects}
1 0.000 0.000 0.002 0.002 ClientDB.py:1000(_AddFiles)
4 0.000 0.000 0.000 0.000 {wx._core_.new_PyEvent}
1 0.000 0.000 0.000 0.000 threading.py:373(notify)
1 0.000 0.000 0.188 0.188 HydrusDB.py:180(_ProcessJob)
1 0.000 0.000 0.002 0.002 HydrusFileHandling.py:126(GetExtraHashesFromPath)
1 0.000 0.000 0.000 0.000 {method 'write' of 'file' objects}
2 0.000 0.000 0.000 0.000 JpegImagePlugin.py:287(_open)
1 0.000 0.000 0.001 0.001 ClientDB.py:1621(_DeleteFiles)
3 0.000 0.000 0.000 0.000 HydrusFileHandling.py:201(GetMime)
7 0.000 0.000 0.000 0.000 {method 'join' of 'str' objects}
2 0.000 0.000 0.024 0.012 ImageFile.py:124(load)
1 0.000 0.000 0.000 0.000 {nt.stat}
1 0.000 0.000 0.000 0.000 HydrusImageHandling.py:130(GetImageProperties)
2 0.000 0.000 0.001 0.000 ClientDB.py:5338(_UpdateAutocompleteTagCacheFromFiles)
1 0.000 0.000 0.028 0.028 Image.py:1498(resize)
11 0.000 0.000 0.000 0.000 {range}
2 0.000 0.000 0.000 0.000 {method 'sum' of 'numpy.ndarray' objects}
2 0.000 0.000 0.000 0.000 ImageFile.py:81(__init__)
1 0.000 0.000 0.000 0.000 HydrusData.py:1708(GetType)
8 0.000 0.000 0.000 0.000 {method 'seek' of 'file' objects}
1 0.000 0.000 0.001 0.001 ClientDB.py:5228(_SyncFileToTagArchive)
37 0.000 0.000 0.000 0.000 {method 'append' of 'list' objects}
6 0.000 0.000 0.000 0.000 JpegImagePlugin.py:60(APP)
1 0.000 0.000 0.001 0.001 threading.py:576(set)
1 0.000 0.000 0.000 0.000 {PIL._imaging.new}
2 0.000 0.000 0.001 0.000 JpegImagePlugin.py:708(jpeg_factory)
7 0.000 0.000 0.000 0.000 HydrusData.py:890(SplayListForDB)
1 0.000 0.000 0.003 0.003 Image.py:1592(save)
1 0.000 0.000 0.000 0.000 {method 'acquire' of 'thread.lock' objects}
1 0.000 0.000 0.000 0.000 threading.py:300(_is_owned)
1 0.000 0.000 0.000 0.000 HydrusTags.py:59(FilterNamespaces)
1 0.000 0.000 0.000 0.000 ClientDB.py:3631(_GetServices)
15 0.000 0.000 0.000 0.000 HydrusData.py:890(<genexpr>)
6 0.000 0.000 0.000 0.000 {method 'fetchall' of 'sqlite3.Cursor' objects}
8 0.000 0.000 0.000 0.000 {method 'fetchone' of 'sqlite3.Cursor' objects}
1 0.000 0.000 0.000 0.000 {nt.lstat}
1 0.000 0.000 0.003 0.003 ImageFile.py:442(_save)
1 0.000 0.000 0.000 0.000 HydrusFileHandling.py:190(GetHashFromPath)
4 0.000 0.000 0.003 0.001 HydrusController.py:84(pub)
9 0.000 0.000 0.000 0.000 {hasattr}
1 0.000 0.000 0.000 0.000 JpegImagePlugin.py:337(draft)
1 0.000 0.000 0.003 0.003 JpegImagePlugin.py:560(_save)
4 0.000 0.000 0.001 0.000 _core.py:7590(_SetSelf)
4 0.000 0.000 0.000 0.000 _core.py:4972(SetEventType)
2 0.000 0.000 0.000 0.000 ClientDB.py:2393(_GetHashId)
4 0.000 0.000 0.002 0.001 ClientController.py:588(NotifyPubSubs)
2 0.000 0.000 0.001 0.000 Image.py:2260(_open_core)
1 0.000 0.000 0.000 0.000 {method 'convert' of 'ImagingCore' objects}
2 0.000 0.000 0.000 0.000 JpegImagePlugin.py:134(SOF)
2 0.000 0.000 0.000 0.000 Image.py:511(_new)
4 0.000 0.000 0.000 0.000 Image.py:491(__init__)
1 0.000 0.000 0.000 0.000 threading.py:288(__exit__)
2 0.000 0.000 0.000 0.000 JpegImagePlugin.py:182(DQT)
4 0.000 0.000 0.000 0.000 _core.py:8421(GetApp)
42 0.000 0.000 0.000 0.000 _binary.py:52(i16be)
4 0.000 0.000 0.001 0.000 _core.py:8403(PostEvent)
6 0.000 0.000 0.000 0.000 ntpath.py:96(splitdrive)
1 0.000 0.000 0.000 0.000 HydrusImageHandling.py:18(ConvertToPngIfBmp)
4 0.000 0.000 0.000 0.000 {method 'digest' of '_hashlib.HASH' objects}
2 0.000 0.000 0.000 0.000 JpegImagePlugin.py:441(_getmp)
2 0.000 0.000 0.001 0.000 Image.py:2218(open)
1 0.000 0.000 0.000 0.000 threading.py:285(__enter__)
4 0.000 0.000 0.000 0.000 {wx._core_.PyEvent_swiginit}
1 0.000 0.000 0.032 0.032 HydrusFileHandling.py:64(GenerateThumbnail)
2 0.000 0.000 0.000 0.000 ClientDB.py:3595(_GetService)
4 0.000 0.000 0.000 0.000 HydrusPaths.py:178(ReadFileLikeAsBlocks)
1 0.000 0.000 0.000 0.000 HydrusTagArchive.py:229(GetTags)
1 0.000 0.000 0.028 0.028 Image.py:1738(thumbnail)
1 0.000 0.000 0.183 0.183 ClientDB.py:6763(_Write)
1 0.000 0.000 0.000 0.000 HydrusTagArchive.py:95(_GetHashId)
2 0.000 0.000 0.000 0.000 ntpath.py:63(join)
1 0.000 0.000 0.000 0.000 ClientDB.py:4193(_InboxFiles)
1 0.000 0.000 0.000 0.000 threading.py:400(notifyAll)
4 0.000 0.000 0.000 0.000 JpegImagePlugin.py:55(Skip)
48 0.000 0.000 0.000 0.000 _binary.py:17(i8)
2 0.000 0.000 0.000 0.000 numeric.py:406(asarray)
2 0.000 0.000 0.000 0.000 _methods.py:31(_sum)
10 0.000 0.000 0.000 0.000 {isinstance}
1 0.000 0.000 0.000 0.000 {numpy.core.multiarray.copyto}
4 0.000 0.000 0.000 0.000 HydrusDB.py:236(pub_after_commit)
1 0.000 0.000 0.032 0.032 HydrusFileHandling.py:45(SaveThumbnailToStream)
1 0.000 0.000 0.001 0.001 HydrusFileHandling.py:148(GetFileInfo)
1 0.000 0.000 0.000 0.000 Image.py:401(_getdecoder)
42 0.000 0.000 0.000 0.000 {_struct.unpack}
5 0.000 0.000 0.000 0.000 Image.py:739(load)
1 0.000 0.000 0.001 0.001 HydrusData.py:1712(PutResult)
34 0.000 0.000 0.000 0.000 {len}
3 0.000 0.000 0.000 0.000 {method 'encode' of 'str' objects}
1 0.000 0.000 0.000 0.000 Image.py:784(convert)
1 0.000 0.000 0.000 0.000 numeric.py:141(ones)
1 0.000 0.000 0.000 0.000 {numpy.core.multiarray.result_type}
3 0.000 0.000 0.000 0.000 hex_codec.py:13(hex_encode)
1 0.000 0.000 0.000 0.000 {numpy.core.multiarray.empty}
4 0.000 0.000 0.000 0.000 Image.py:617(__getattr__)
4 0.000 0.000 0.000 0.000 ClientDB.py:5350(<genexpr>)
14 0.000 0.000 0.000 0.000 ImageFile.py:496(_safe_read)
1 0.000 0.000 0.028 0.028 HydrusImageHandling.py:59(EfficientlyThumbnailPILImage)
1 0.000 0.000 0.000 0.000 HydrusTagArchive.py:190(GetHashType)
1 0.000 0.000 0.000 0.000 HydrusImageHandling.py:148(GetResolutionAndNumFrames)
1 0.000 0.000 0.000 0.000 ClientFiles.py:150(GetExpectedThumbnailPath)
5 0.000 0.000 0.000 0.000 _util.py:7(isPath)
1 0.000 0.000 0.000 0.000 ClientCaches.py:324(GetExpectedFilePath)
2 0.000 0.000 0.000 0.000 {method 'sort' of 'list' objects}
1 0.000 0.000 0.000 0.000 ClientDB.py:6804(pub_content_updates_after_commit)
1 0.000 0.000 0.000 0.000 Image.py:418(_getencoder)
1 0.000 0.000 0.000 0.000 _methods.py:37(_any)
1 0.000 0.000 0.188 0.188 <string>:1(<module>)
1 0.000 0.000 0.000 0.000 ClientFiles.py:142(GetExpectedFilePath)
1 0.000 0.000 0.000 0.000 HydrusData.py:629(GetNow)
1 0.000 0.000 0.000 0.000 HydrusData.py:1555(__init__)
1 0.000 0.000 0.000 0.000 genericpath.py:93(_splitext)
2 0.000 0.000 0.001 0.000 HydrusImageHandling.py:73(GeneratePILImage)
1 0.000 0.000 0.000 0.000 ClientDB.py:1164(<setcomp>)
1 0.000 0.000 0.000 0.000 ClientDB.py:2380(_GetHash)
1 0.000 0.000 0.000 0.000 {method 'write' of 'cStringIO.StringO' objects}
2 0.000 0.000 0.000 0.000 JpegImagePlugin.py:275(_accept)
1 0.000 0.000 0.000 0.000 {_hashlib.openssl_md5}
1 0.000 0.000 0.000 0.000 ClientCaches.py:199(_GetLocation)
1 0.000 0.000 0.000 0.000 HydrusTags.py:143(CleanTags)
2 0.000 0.000 0.000 0.000 ClientDB.py:5348(<setcomp>)
2 0.000 0.000 0.000 0.000 JpegImagePlugin.py:393(_getmp)
3 0.000 0.000 0.000 0.000 {binascii.b2a_hex}
1 0.000 0.000 0.000 0.000 genericpath.py:23(exists)
1 0.000 0.000 0.000 0.000 {PIL._imaging.jpeg_encoder}
2 0.000 0.000 0.000 0.000 {method 'replace' of 'unicode' objects}
2 0.000 0.000 0.000 0.000 {method 'intersection' of 'set' objects}
1 0.000 0.000 0.000 0.000 HydrusDB.py:111(_GetRowCount)
2 0.000 0.000 0.000 0.000 {method 'copy' of 'dict' objects}
1 0.000 0.000 0.000 0.000 ImageFile.py:251(load_prepare)
1 0.000 0.000 0.000 0.000 ClientDB.py:3014(_GetHashIdStatus)
2 0.000 0.000 0.000 0.000 Image.py:2204(_decompression_bomb_check)
5 0.000 0.000 0.000 0.000 {max}
2 0.000 0.000 0.000 0.000 ClientData.py:958(GetServiceType)
1 0.000 0.000 0.000 0.000 {PIL._imaging.jpeg_decoder}
1 0.000 0.000 0.000 0.000 ClientData.py:730(ToTuple)
2 0.000 0.000 0.000 0.000 ClientData.py:948(GetServiceKey)
1 0.000 0.000 0.000 0.000 ClientDB.py:1627(<setcomp>)
2 0.000 0.000 0.000 0.000 {divmod}
2 0.000 0.000 0.000 0.000 ClientData.py:942(GetInfo)
2 0.000 0.000 0.000 0.000 {method 'items' of 'dict' objects}
1 0.000 0.000 0.000 0.000 ntpath.py:199(splitext)
3 0.000 0.000 0.000 0.000 {method 'startswith' of 'str' objects}
4 0.000 0.000 0.000 0.000 {method 'pixel_access' of 'ImagingCore' objects}
1 0.000 0.000 0.000 0.000 {_hashlib.openssl_sha256}
3 0.000 0.000 0.000 0.000 {method 'rfind' of 'str' objects}
48 0.000 0.000 0.000 0.000 {ord}
1 0.000 0.000 0.000 0.000 {_hashlib.openssl_sha512}
1 0.000 0.000 0.000 0.000 {method 'remove' of 'list' objects}
4 0.000 0.000 0.000 0.000 {callable}
2 0.000 0.000 0.000 0.000 {numpy.core.multiarray.array}
4 0.000 0.000 0.000 0.000 {method 'replace' of 'str' objects}
2 0.000 0.000 0.000 0.000 TiffImagePlugin.py:214(_accept)
1 0.000 0.000 0.000 0.000 threading.py:64(_note)
2 0.000 0.000 0.000 0.000 ClientDB.py:4195(<genexpr>)
3 0.000 0.000 0.000 0.000 Image.py:341(preinit)
1 0.000 0.000 0.000 0.000 {_hashlib.openssl_sha1}
6 0.000 0.000 0.000 0.000 {method 'update' of 'set' objects}
1 0.000 0.000 0.000 0.000 HydrusData.py:1710(IsSynchronous)
2 0.000 0.000 0.000 0.000 BmpImagePlugin.py:54(_accept)
1 0.000 0.000 0.000 0.000 {method '__exit__' of 'thread.lock' objects}
1 0.000 0.000 0.000 0.000 {method 'read' of 'cStringIO.StringO' objects}
1 0.000 0.000 0.000 0.000 {method 'setimage' of 'ImagingEncoder' objects}
1 0.000 0.000 0.000 0.000 {sum}
1 0.000 0.000 0.000 0.000 {method 'setimage' of 'ImagingDecoder' objects}
9 0.000 0.000 0.000 0.000 {method 'get' of 'dict' objects}
2 0.000 0.000 0.000 0.000 GifImagePlugin.py:45(_accept)
2 0.000 0.000 0.000 0.000 {getattr}
1 0.000 0.000 0.000 0.000 Image.py:1671(seek)
1 0.000 0.000 0.000 0.000 {time.time}
2 0.000 0.000 0.000 0.000 {method 'upper' of 'str' objects}
2 0.000 0.000 0.000 0.000 ImageFile.py:69(_tilesort)
1 0.000 0.000 0.000 0.000 HydrusData.py:1674(GetAction)
1 0.000 0.000 0.000 0.000 {method 'add' of 'set' objects}
1 0.000 0.000 0.000 0.000 {method 'cleanup' of 'ImagingDecoder' objects}
1 0.000 0.000 0.000 0.000 {method 'seek' of 'cStringIO.StringO' objects}
1 0.000 0.000 0.000 0.000 {method 'release' of 'thread.lock' objects}
1 0.000 0.000 0.000 0.000 _util.py:4(isStringType)
1 0.000 0.000 0.000 0.000 {cStringIO.StringIO}
1 0.000 0.000 0.000 0.000 ClientController.py:387(GetClientFilesManager)
1 0.000 0.000 0.000 0.000 JpegImagePlugin.py:604(validate_qtables)
1 0.000 0.000 0.000 0.000 HydrusData.py:1678(GetKWArgs)
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
1 0.000 0.000 0.000 0.000 {method 'flush' of 'cStringIO.StringO' objects}
1 0.000 0.000 0.000 0.000 {method 'close' of 'cStringIO.StringO' objects}
1 0.000 0.000 0.000 0.000 HydrusData.py:1676(GetArgs)
1 0.000 0.000 0.000 0.000 ImageFile.py:260(load_end)
1 0.000 0.000 0.000 0.000 {method 'lower' of 'str' objects}
1 0.000 0.000 0.000 0.000 {method 'cleanup' of 'ImagingEncoder' objects}
2015/12/15 14:41:14: Callers
Ordered by: internal time
Function was called by...
ncalls tottime cumtime
{dct} <- 1 0.077 0.077 ClientImageHandling.py:66(GeneratePerceptualHash)
function_base.py:492(average) <- 1 0.031 0.032 ClientImageHandling.py:66(GeneratePerceptualHash)
{method 'decode' of 'ImagingDecoder' objects} <- 1 0.023 0.023 ImageFile.py:124(load)
{cvtColor} <- 1 0.015 0.015 ClientImageHandling.py:66(GeneratePerceptualHash)
{method 'execute' of 'sqlite3.Cursor' objects} <- 6 0.000 0.000 ClientDB.py:1000(_AddFiles)
2 0.009 0.009 ClientDB.py:1147(_AddThumbnails)
5 0.000 0.000 ClientDB.py:1621(_DeleteFiles)
1 0.000 0.000 ClientDB.py:2380(_GetHash)
2 0.000 0.000 ClientDB.py:2393(_GetHashId)
1 0.000 0.000 ClientDB.py:3014(_GetHashIdStatus)
1 0.000 0.000 ClientDB.py:3631(_GetServices)
1 0.000 0.000 ClientDB.py:4055(_ImportFile)
1 0.000 0.000 ClientDB.py:5228(_SyncFileToTagArchive)
4 0.000 0.000 ClientDB.py:5338(_UpdateAutocompleteTagCacheFromFiles)
2 0.001 0.001 HydrusDB.py:180(_ProcessJob)
1 0.000 0.000 HydrusTagArchive.py:95(_GetHashId)
1 0.000 0.000 HydrusTagArchive.py:190(GetHashType)
{resize} <- 1 0.007 0.007 ClientImageHandling.py:66(GeneratePerceptualHash)
{method 'resize' of 'ImagingCore' objects} <- 1 0.004 0.004 Image.py:1498(resize)
ClientImageHandling.py:66(GeneratePerceptualHash) <- 1 0.004 0.136 ClientDB.py:1147(_AddThumbnails)
{method 'encode' of 'ImagingEncoder' objects} <- 1 0.003 0.003 ImageFile.py:442(_save)
{method 'update' of '_hashlib.HASH' objects} <- 3 0.001 0.001 HydrusFileHandling.py:126(GetExtraHashesFromPath)
1 0.000 0.000 HydrusFileHandling.py:190(GetHashFromPath)
{imread} <- 1 0.001 0.001 ClientImageHandling.py:66(GeneratePerceptualHash)
{open} <- 1 0.000 0.000 ClientDB.py:1147(_AddThumbnails)
1 0.000 0.000 HydrusFileHandling.py:126(GetExtraHashesFromPath)
1 0.000 0.000 HydrusFileHandling.py:190(GetHashFromPath)
3 0.000 0.000 HydrusFileHandling.py:201(GetMime)
1 0.000 0.000 HydrusImageHandling.py:18(ConvertToPngIfBmp)
2 0.000 0.000 Image.py:2218(open)
{wx._core_.PostEvent} <- 4 0.001 0.001 _core.py:8403(PostEvent)
{method 'any' of 'numpy.generic' objects} <- 1 0.001 0.001 function_base.py:492(average)
{wx._core_.PyEvent__SetSelf} <- 4 0.000 0.000 _core.py:7590(_SetSelf)
{method 'read' of 'file' objects} <- 3 0.000 0.000 HydrusFileHandling.py:201(GetMime)
1 0.000 0.000 HydrusImageHandling.py:18(ConvertToPngIfBmp)
4 0.000 0.000 HydrusPaths.py:178(ReadFileLikeAsBlocks)
2 0.000 0.000 Image.py:2218(open)
1 0.000 0.000 ImageFile.py:124(load)
14 0.000 0.000 ImageFile.py:496(_safe_read)
4 0.000 0.000 JpegImagePlugin.py:55(Skip)
6 0.000 0.000 JpegImagePlugin.py:60(APP)
2 0.000 0.000 JpegImagePlugin.py:134(SOF)
2 0.000 0.000 JpegImagePlugin.py:182(DQT)
32 0.000 0.000 JpegImagePlugin.py:287(_open)
{method 'executemany' of 'sqlite3.Cursor' objects} <- 1 0.000 0.000 ClientDB.py:1000(_AddFiles)
1 0.000 0.000 ClientDB.py:1621(_DeleteFiles)
1 0.000 0.000 ClientDB.py:4193(_InboxFiles)
6 0.000 0.000 ClientDB.py:5338(_UpdateAutocompleteTagCacheFromFiles)
{wx._core_.Event_SetEventType} <- 4 0.000 0.000 _core.py:4972(SetEventType)
{method '__enter__' of 'thread.lock' objects} <- 1 0.000 0.000 threading.py:285(__enter__)
_core.py:16750(CallAfter) <- 4 0.000 0.002 ClientController.py:588(NotifyPubSubs)
ClientDB.py:1147(_AddThumbnails) <- 1 0.000 0.145 ClientDB.py:4055(_ImportFile)
_core.py:7583(__init__) <- 4 0.000 0.001 _core.py:16750(CallAfter)
HydrusPubSub.py:129(pub) <- 4 0.000 0.003 HydrusController.py:84(pub)
{wx._core_.GetApp} <- 4 0.000 0.000 _core.py:8421(GetApp)
ClientDB.py:4055(_ImportFile) <- 1 0.000 0.183 ClientDB.py:6763(_Write)
{method 'reduce' of 'numpy.ufunc' objects} <- 2 0.000 0.000 _methods.py:31(_sum)
1 0.000 0.000 _methods.py:37(_any)
ClientDB.py:1000(_AddFiles) <- 1 0.000 0.002 ClientDB.py:4055(_ImportFile)
{wx._core_.new_PyEvent} <- 4 0.000 0.000 _core.py:7583(__init__)
threading.py:373(notify) <- 1 0.000 0.000 threading.py:400(notifyAll)
HydrusDB.py:180(_ProcessJob) <- 1 0.000 0.188 <string>:1(<module>)
HydrusFileHandling.py:126(GetExtraHashesFromPath) <- 1 0.000 0.002 ClientDB.py:4055(_ImportFile)
{method 'write' of 'file' objects} <- 1 0.000 0.000 ClientDB.py:1147(_AddThumbnails)
JpegImagePlugin.py:287(_open) <- 2 0.000 0.000 ImageFile.py:81(__init__)
ClientDB.py:1621(_DeleteFiles) <- 1 0.000 0.001 ClientDB.py:1000(_AddFiles)
HydrusFileHandling.py:201(GetMime) <- 1 0.000 0.000 ClientDB.py:4055(_ImportFile)
1 0.000 0.000 HydrusFileHandling.py:64(GenerateThumbnail)
1 0.000 0.000 HydrusFileHandling.py:148(GetFileInfo)
{method 'join' of 'str' objects} <- 7 0.000 0.000 HydrusData.py:890(SplayListForDB)
ImageFile.py:124(load) <- 1 0.000 0.000 Image.py:784(convert)
1 0.000 0.024 Image.py:1498(resize)
{nt.stat} <- 1 0.000 0.000 genericpath.py:23(exists)
HydrusImageHandling.py:130(GetImageProperties) <- 1 0.000 0.000 HydrusFileHandling.py:148(GetFileInfo)
ClientDB.py:5338(_UpdateAutocompleteTagCacheFromFiles) <- 1 0.000 0.000 ClientDB.py:1000(_AddFiles)
1 0.000 0.000 ClientDB.py:1621(_DeleteFiles)
Image.py:1498(resize) <- 1 0.000 0.028 Image.py:1738(thumbnail)
{range} <- 9 0.000 0.000 ClientImageHandling.py:66(GeneratePerceptualHash)
2 0.000 0.000 JpegImagePlugin.py:134(SOF)
{method 'sum' of 'numpy.ndarray' objects} <- 2 0.000 0.000 function_base.py:492(average)
ImageFile.py:81(__init__) <- 2 0.000 0.000 JpegImagePlugin.py:708(jpeg_factory)
HydrusData.py:1708(GetType) <- 1 0.000 0.000 HydrusDB.py:180(_ProcessJob)
{method 'seek' of 'file' objects} <- 3 0.000 0.000 HydrusFileHandling.py:201(GetMime)
2 0.000 0.000 Image.py:2218(open)
2 0.000 0.000 Image.py:2260(_open_core)
1 0.000 0.000 ImageFile.py:124(load)
ClientDB.py:5228(_SyncFileToTagArchive) <- 1 0.000 0.001 ClientDB.py:4055(_ImportFile)
{method 'append' of 'list' objects} <- 5 0.000 0.000 ClientDB.py:1000(_AddFiles)
4 0.000 0.000 ClientDB.py:1621(_DeleteFiles)
8 0.000 0.000 ClientImageHandling.py:66(GeneratePerceptualHash)
4 0.000 0.000 HydrusDB.py:236(pub_after_commit)
4 0.000 0.000 HydrusPubSub.py:129(pub)
6 0.000 0.000 JpegImagePlugin.py:60(APP)
6 0.000 0.000 JpegImagePlugin.py:134(SOF)
JpegImagePlugin.py:60(APP) <- 6 0.000 0.000 JpegImagePlugin.py:287(_open)
threading.py:576(set) <- 1 0.000 0.001 HydrusData.py:1712(PutResult)
{PIL._imaging.new} <- 1 0.000 0.000 ImageFile.py:251(load_prepare)
JpegImagePlugin.py:708(jpeg_factory) <- 2 0.000 0.001 Image.py:2260(_open_core)
HydrusData.py:890(SplayListForDB) <- 2 0.000 0.000 ClientDB.py:1000(_AddFiles)
2 0.000 0.000 ClientDB.py:1621(_DeleteFiles)
1 0.000 0.000 ClientDB.py:3631(_GetServices)
2 0.000 0.000 ClientDB.py:5338(_UpdateAutocompleteTagCacheFromFiles)
Image.py:1592(save) <- 1 0.000 0.003 HydrusFileHandling.py:45(SaveThumbnailToStream)
{method 'acquire' of 'thread.lock' objects} <- 1 0.000 0.000 threading.py:300(_is_owned)
threading.py:300(_is_owned) <- 1 0.000 0.000 threading.py:373(notify)
HydrusTags.py:59(FilterNamespaces) <- 1 0.000 0.000 ClientDB.py:5228(_SyncFileToTagArchive)
ClientDB.py:3631(_GetServices) <- 1 0.000 0.000 ClientDB.py:4055(_ImportFile)
HydrusData.py:890(<genexpr>) <- 15 0.000 0.000 {method 'join' of 'str' objects}
{method 'fetchall' of 'sqlite3.Cursor' objects} <- 1 0.000 0.000 ClientDB.py:1000(_AddFiles)
1 0.000 0.000 ClientDB.py:1621(_DeleteFiles)
4 0.000 0.000 ClientDB.py:5338(_UpdateAutocompleteTagCacheFromFiles)
{method 'fetchone' of 'sqlite3.Cursor' objects} <- 1 0.000 0.000 ClientDB.py:1000(_AddFiles)
1 0.000 0.000 ClientDB.py:2380(_GetHash)
2 0.000 0.000 ClientDB.py:2393(_GetHashId)
1 0.000 0.000 ClientDB.py:3014(_GetHashIdStatus)
1 0.000 0.000 ClientDB.py:5228(_SyncFileToTagArchive)
1 0.000 0.000 HydrusTagArchive.py:95(_GetHashId)
1 0.000 0.000 HydrusTagArchive.py:190(GetHashType)
{nt.lstat} <- 1 0.000 0.000 HydrusFileHandling.py:148(GetFileInfo)
ImageFile.py:442(_save) <- 1 0.000 0.003 JpegImagePlugin.py:560(_save)
HydrusFileHandling.py:190(GetHashFromPath) <- 1 0.000 0.000 ClientDB.py:4055(_ImportFile)
HydrusController.py:84(pub) <- 4 0.000 0.003 HydrusDB.py:180(_ProcessJob)
{hasattr} <- 1 0.000 0.000 Image.py:1592(save)
2 0.000 0.000 ImageFile.py:124(load)
2 0.000 0.000 ImageFile.py:442(_save)
4 0.000 0.000 _core.py:16750(CallAfter)
JpegImagePlugin.py:337(draft) <- 1 0.000 0.000 Image.py:1738(thumbnail)
JpegImagePlugin.py:560(_save) <- 1 0.000 0.003 Image.py:1592(save)
_core.py:7590(_SetSelf) <- 4 0.000 0.001 _core.py:7583(__init__)
_core.py:4972(SetEventType) <- 4 0.000 0.000 _core.py:16750(CallAfter)
ClientDB.py:2393(_GetHashId) <- 1 0.000 0.000 ClientDB.py:1147(_AddThumbnails)
1 0.000 0.000 ClientDB.py:4055(_ImportFile)
ClientController.py:588(NotifyPubSubs) <- 4 0.000 0.002 HydrusPubSub.py:129(pub)
Image.py:2260(_open_core) <- 2 0.000 0.001 Image.py:2218(open)
{method 'convert' of 'ImagingCore' objects} <- 1 0.000 0.000 Image.py:784(convert)
JpegImagePlugin.py:134(SOF) <- 2 0.000 0.000 JpegImagePlugin.py:287(_open)
Image.py:511(_new) <- 1 0.000 0.000 Image.py:784(convert)
1 0.000 0.000 Image.py:1498(resize)
Image.py:491(__init__) <- 2 0.000 0.000 Image.py:511(_new)
2 0.000 0.000 ImageFile.py:81(__init__)
threading.py:288(__exit__) <- 1 0.000 0.000 threading.py:576(set)
JpegImagePlugin.py:182(DQT) <- 2 0.000 0.000 JpegImagePlugin.py:287(_open)
_core.py:8421(GetApp) <- 4 0.000 0.000 _core.py:16750(CallAfter)
_binary.py:52(i16be) <- 4 0.000 0.000 JpegImagePlugin.py:55(Skip)
14 0.000 0.000 JpegImagePlugin.py:60(APP)
6 0.000 0.000 JpegImagePlugin.py:134(SOF)
2 0.000 0.000 JpegImagePlugin.py:182(DQT)
16 0.000 0.000 JpegImagePlugin.py:287(_open)
_core.py:8403(PostEvent) <- 4 0.000 0.001 _core.py:16750(CallAfter)
ntpath.py:96(splitdrive) <- 6 0.000 0.000 ntpath.py:63(join)
HydrusImageHandling.py:18(ConvertToPngIfBmp) <- 1 0.000 0.000 ClientDB.py:4055(_ImportFile)
{method 'digest' of '_hashlib.HASH' objects} <- 3 0.000 0.000 HydrusFileHandling.py:126(GetExtraHashesFromPath)
1 0.000 0.000 HydrusFileHandling.py:190(GetHashFromPath)
JpegImagePlugin.py:441(_getmp) <- 2 0.000 0.000 JpegImagePlugin.py:393(_getmp)
Image.py:2218(open) <- 2 0.000 0.001 HydrusImageHandling.py:73(GeneratePILImage)
threading.py:285(__enter__) <- 1 0.000 0.000 threading.py:576(set)
{wx._core_.PyEvent_swiginit} <- 4 0.000 0.000 _core.py:7583(__init__)
HydrusFileHandling.py:64(GenerateThumbnail) <- 1 0.000 0.032 ClientDB.py:4055(_ImportFile)
ClientDB.py:3595(_GetService) <- 2 0.000 0.000 ClientDB.py:3631(_GetServices)
HydrusPaths.py:178(ReadFileLikeAsBlocks) <- 2 0.000 0.000 HydrusFileHandling.py:126(GetExtraHashesFromPath)
2 0.000 0.000 HydrusFileHandling.py:190(GetHashFromPath)
HydrusTagArchive.py:229(GetTags) <- 1 0.000 0.000 ClientDB.py:5228(_SyncFileToTagArchive)
Image.py:1738(thumbnail) <- 1 0.000 0.028 HydrusImageHandling.py:59(EfficientlyThumbnailPILImage)
ClientDB.py:6763(_Write) <- 1 0.000 0.183 HydrusDB.py:180(_ProcessJob)
HydrusTagArchive.py:95(_GetHashId) <- 1 0.000 0.000 HydrusTagArchive.py:229(GetTags)
ntpath.py:63(join) <- 1 0.000 0.000 ClientFiles.py:142(GetExpectedFilePath)
1 0.000 0.000 ClientFiles.py:150(GetExpectedThumbnailPath)
ClientDB.py:4193(_InboxFiles) <- 1 0.000 0.000 ClientDB.py:4055(_ImportFile)
threading.py:400(notifyAll) <- 1 0.000 0.000 threading.py:576(set)
JpegImagePlugin.py:55(Skip) <- 4 0.000 0.000 JpegImagePlugin.py:287(_open)
_binary.py:17(i8) <- 4 0.000 0.000 JpegImagePlugin.py:60(APP)
22 0.000 0.000 JpegImagePlugin.py:134(SOF)
4 0.000 0.000 JpegImagePlugin.py:182(DQT)
18 0.000 0.000 JpegImagePlugin.py:287(_open)
numeric.py:406(asarray) <- 2 0.000 0.000 function_base.py:492(average)
_methods.py:31(_sum) <- 2 0.000 0.000 {method 'sum' of 'numpy.ndarray' objects}
{isinstance} <- 1 0.000 0.000 Image.py:401(_getdecoder)
1 0.000 0.000 Image.py:418(_getencoder)
1 0.000 0.000 JpegImagePlugin.py:560(_save)
1 0.000 0.000 _util.py:4(isStringType)
5 0.000 0.000 _util.py:7(isPath)
1 0.000 0.000 function_base.py:492(average)
{numpy.core.multiarray.copyto} <- 1 0.000 0.000 numeric.py:141(ones)
HydrusDB.py:236(pub_after_commit) <- 1 0.000 0.000 ClientDB.py:1147(_AddThumbnails)
1 0.000 0.000 ClientDB.py:1621(_DeleteFiles)
2 0.000 0.000 ClientDB.py:6804(pub_content_updates_after_commit)
HydrusFileHandling.py:45(SaveThumbnailToStream) <- 1 0.000 0.032 HydrusFileHandling.py:64(GenerateThumbnail)
HydrusFileHandling.py:148(GetFileInfo) <- 1 0.000 0.001 ClientDB.py:4055(_ImportFile)
Image.py:401(_getdecoder) <- 1 0.000 0.000 ImageFile.py:124(load)
{_struct.unpack} <- 42 0.000 0.000 _binary.py:52(i16be)
Image.py:739(load) <- 1 0.000 0.000 Image.py:1592(save)
3 0.000 0.000 ImageFile.py:124(load)
1 0.000 0.000 ImageFile.py:442(_save)
HydrusData.py:1712(PutResult) <- 1 0.000 0.001 HydrusDB.py:180(_ProcessJob)
{len} <- 4 0.000 0.000 ClientDB.py:1000(_AddFiles)
4 0.000 0.000 ClientDB.py:1621(_DeleteFiles)
1 0.000 0.000 ClientDB.py:5228(_SyncFileToTagArchive)
1 0.000 0.000 ImageFile.py:124(load)
2 0.000 0.000 JpegImagePlugin.py:134(SOF)
10 0.000 0.000 JpegImagePlugin.py:182(DQT)
1 0.000 0.000 JpegImagePlugin.py:337(draft)
1 0.000 0.000 JpegImagePlugin.py:560(_save)
3 0.000 0.000 hex_codec.py:13(hex_encode)
6 0.000 0.000 ntpath.py:96(splitdrive)
1 0.000 0.000 threading.py:400(notifyAll)
{method 'encode' of 'str' objects} <- 1 0.000 0.000 ClientCaches.py:199(_GetLocation)
1 0.000 0.000 ClientFiles.py:142(GetExpectedFilePath)
1 0.000 0.000 ClientFiles.py:150(GetExpectedThumbnailPath)
Image.py:784(convert) <- 1 0.000 0.000 HydrusFileHandling.py:45(SaveThumbnailToStream)
numeric.py:141(ones) <- 1 0.000 0.000 ClientImageHandling.py:66(GeneratePerceptualHash)
{numpy.core.multiarray.result_type} <- 1 0.000 0.000 function_base.py:492(average)
hex_codec.py:13(hex_encode) <- 3 0.000 0.000 {method 'encode' of 'str' objects}
{numpy.core.multiarray.empty} <- 1 0.000 0.000 numeric.py:141(ones)
Image.py:617(__getattr__) <- 3 0.000 0.000 ImageFile.py:124(load)
1 0.000 0.000 {hasattr}
ClientDB.py:5350(<genexpr>) <- 4 0.000 0.000 {method 'executemany' of 'sqlite3.Cursor' objects}
ImageFile.py:496(_safe_read) <- 4 0.000 0.000 JpegImagePlugin.py:55(Skip)
6 0.000 0.000 JpegImagePlugin.py:60(APP)
2 0.000 0.000 JpegImagePlugin.py:134(SOF)
2 0.000 0.000 JpegImagePlugin.py:182(DQT)
HydrusImageHandling.py:59(EfficientlyThumbnailPILImage) <- 1 0.000 0.028 HydrusFileHandling.py:45(SaveThumbnailToStream)
HydrusTagArchive.py:190(GetHashType) <- 1 0.000 0.000 ClientDB.py:5228(_SyncFileToTagArchive)
HydrusImageHandling.py:148(GetResolutionAndNumFrames) <- 1 0.000 0.000 HydrusImageHandling.py:130(GetImageProperties)
ClientFiles.py:150(GetExpectedThumbnailPath) <- 1 0.000 0.000 ClientDB.py:1147(_AddThumbnails)
_util.py:7(isPath) <- 1 0.000 0.000 Image.py:1592(save)
2 0.000 0.000 Image.py:2218(open)
2 0.000 0.000 ImageFile.py:81(__init__)
ClientCaches.py:324(GetExpectedFilePath) <- 1 0.000 0.000 ClientDB.py:4055(_ImportFile)
{method 'sort' of 'list' objects} <- 1 0.000 0.000 ImageFile.py:124(load)
1 0.000 0.000 ImageFile.py:442(_save)
ClientDB.py:6804(pub_content_updates_after_commit) <- 1 0.000 0.000 ClientDB.py:4055(_ImportFile)
Image.py:418(_getencoder) <- 1 0.000 0.000 ImageFile.py:442(_save)
_methods.py:37(_any) <- 1 0.000 0.000 {method 'any' of 'numpy.generic' objects}
<string>:1(<module>) <-
ClientFiles.py:142(GetExpectedFilePath) <- 1 0.000 0.000 ClientCaches.py:324(GetExpectedFilePath)
HydrusData.py:629(GetNow) <- 1 0.000 0.000 ClientDB.py:4055(_ImportFile)
HydrusData.py:1555(__init__) <- 1 0.000 0.000 ClientDB.py:4055(_ImportFile)
genericpath.py:93(_splitext) <- 1 0.000 0.000 ntpath.py:199(splitext)
HydrusImageHandling.py:73(GeneratePILImage) <- 1 0.000 0.000 HydrusFileHandling.py:64(GenerateThumbnail)
1 0.000 0.000 HydrusImageHandling.py:148(GetResolutionAndNumFrames)
ClientDB.py:1164(<setcomp>) <- 1 0.000 0.000 ClientDB.py:1147(_AddThumbnails)
ClientDB.py:2380(_GetHash) <- 1 0.000 0.000 ClientDB.py:5228(_SyncFileToTagArchive)
{method 'write' of 'cStringIO.StringO' objects} <- 1 0.000 0.000 ImageFile.py:442(_save)
JpegImagePlugin.py:275(_accept) <- 2 0.000 0.000 Image.py:2260(_open_core)
{_hashlib.openssl_md5} <- 1 0.000 0.000 HydrusFileHandling.py:126(GetExtraHashesFromPath)
ClientCaches.py:199(_GetLocation) <- 1 0.000 0.000 ClientCaches.py:324(GetExpectedFilePath)
HydrusTags.py:143(CleanTags) <- 1 0.000 0.000 ClientDB.py:5228(_SyncFileToTagArchive)
ClientDB.py:5348(<setcomp>) <- 2 0.000 0.000 ClientDB.py:5338(_UpdateAutocompleteTagCacheFromFiles)
JpegImagePlugin.py:393(_getmp) <- 2 0.000 0.000 JpegImagePlugin.py:708(jpeg_factory)
{binascii.b2a_hex} <- 3 0.000 0.000 hex_codec.py:13(hex_encode)
genericpath.py:23(exists) <- 1 0.000 0.000 ClientDB.py:4055(_ImportFile)
{PIL._imaging.jpeg_encoder} <- 1 0.000 0.000 Image.py:418(_getencoder)
{method 'replace' of 'unicode' objects} <- 2 0.000 0.000 ntpath.py:96(splitdrive)
{method 'intersection' of 'set' objects} <- 1 0.000 0.000 ClientDB.py:1000(_AddFiles)
1 0.000 0.000 ClientDB.py:1621(_DeleteFiles)
HydrusDB.py:111(_GetRowCount) <- 1 0.000 0.000 ClientDB.py:4193(_InboxFiles)
{method 'copy' of 'dict' objects} <- 2 0.000 0.000 Image.py:511(_new)
ImageFile.py:251(load_prepare) <- 1 0.000 0.000 ImageFile.py:124(load)
ClientDB.py:3014(_GetHashIdStatus) <- 1 0.000 0.000 ClientDB.py:4055(_ImportFile)
Image.py:2204(_decompression_bomb_check) <- 2 0.000 0.000 Image.py:2260(_open_core)
{max} <- 1 0.000 0.000 Image.py:1738(thumbnail)
1 0.000 0.000 ImageFile.py:442(_save)
1 0.000 0.000 JpegImagePlugin.py:337(draft)
1 0.000 0.000 JpegImagePlugin.py:560(_save)
1 0.000 0.000 genericpath.py:93(_splitext)
ClientData.py:958(GetServiceType) <- 2 0.000 0.000 ClientDB.py:3595(_GetService)
{PIL._imaging.jpeg_decoder} <- 1 0.000 0.000 Image.py:401(_getdecoder)
ClientData.py:730(ToTuple) <- 1 0.000 0.000 ClientDB.py:4055(_ImportFile)
ClientData.py:948(GetServiceKey) <- 2 0.000 0.000 ClientDB.py:4055(_ImportFile)
ClientDB.py:1627(<setcomp>) <- 1 0.000 0.000 ClientDB.py:1621(_DeleteFiles)
{divmod} <- 2 0.000 0.000 JpegImagePlugin.py:60(APP)
ClientData.py:942(GetInfo) <- 2 0.000 0.000 ClientDB.py:4055(_ImportFile)
{method 'items' of 'dict' objects} <- 2 0.000 0.000 ClientDB.py:4055(_ImportFile)
ntpath.py:199(splitext) <- 1 0.000 0.000 Image.py:1592(save)
{method 'startswith' of 'str' objects} <- 3 0.000 0.000 HydrusFileHandling.py:201(GetMime)
{method 'pixel_access' of 'ImagingCore' objects} <- 4 0.000 0.000 Image.py:739(load)
{_hashlib.openssl_sha256} <- 1 0.000 0.000 HydrusFileHandling.py:190(GetHashFromPath)
{method 'rfind' of 'str' objects} <- 3 0.000 0.000 genericpath.py:93(_splitext)
{ord} <- 48 0.000 0.000 _binary.py:17(i8)
{_hashlib.openssl_sha512} <- 1 0.000 0.000 HydrusFileHandling.py:126(GetExtraHashesFromPath)
{method 'remove' of 'list' objects} <- 1 0.000 0.000 threading.py:373(notify)
{callable} <- 4 0.000 0.000 _core.py:16750(CallAfter)
{numpy.core.multiarray.array} <- 2 0.000 0.000 numeric.py:406(asarray)
{method 'replace' of 'str' objects} <- 4 0.000 0.000 ntpath.py:96(splitdrive)
TiffImagePlugin.py:214(_accept) <- 2 0.000 0.000 Image.py:2260(_open_core)
threading.py:64(_note) <- 1 0.000 0.000 threading.py:373(notify)
ClientDB.py:4195(<genexpr>) <- 2 0.000 0.000 {method 'executemany' of 'sqlite3.Cursor' objects}
Image.py:341(preinit) <- 1 0.000 0.000 Image.py:1592(save)
2 0.000 0.000 Image.py:2218(open)
{_hashlib.openssl_sha1} <- 1 0.000 0.000 HydrusFileHandling.py:126(GetExtraHashesFromPath)
{method 'update' of 'set' objects} <- 6 0.000 0.000 HydrusTags.py:59(FilterNamespaces)
HydrusData.py:1710(IsSynchronous) <- 1 0.000 0.000 HydrusDB.py:180(_ProcessJob)
BmpImagePlugin.py:54(_accept) <- 2 0.000 0.000 Image.py:2260(_open_core)
{method '__exit__' of 'thread.lock' objects} <- 1 0.000 0.000 threading.py:288(__exit__)
{method 'read' of 'cStringIO.StringO' objects} <- 1 0.000 0.000 HydrusFileHandling.py:64(GenerateThumbnail)
{method 'setimage' of 'ImagingEncoder' objects} <- 1 0.000 0.000 ImageFile.py:442(_save)
{sum} <- 1 0.000 0.000 ClientDB.py:1621(_DeleteFiles)
{method 'setimage' of 'ImagingDecoder' objects} <- 1 0.000 0.000 ImageFile.py:124(load)
{method 'get' of 'dict' objects} <- 9 0.000 0.000 JpegImagePlugin.py:560(_save)
GifImagePlugin.py:45(_accept) <- 2 0.000 0.000 Image.py:2260(_open_core)
{getattr} <- 1 0.000 0.000 Image.py:401(_getdecoder)
1 0.000 0.000 Image.py:418(_getencoder)
Image.py:1671(seek) <- 1 0.000 0.000 HydrusImageHandling.py:148(GetResolutionAndNumFrames)
{time.time} <- 1 0.000 0.000 HydrusData.py:629(GetNow)
{method 'upper' of 'str' objects} <- 2 0.000 0.000 Image.py:1592(save)
ImageFile.py:69(_tilesort) <- 2 0.000 0.000 {method 'sort' of 'list' objects}
HydrusData.py:1674(GetAction) <- 1 0.000 0.000 HydrusDB.py:180(_ProcessJob)
{method 'add' of 'set' objects} <- 1 0.000 0.000 ClientDB.py:1000(_AddFiles)
{method 'cleanup' of 'ImagingDecoder' objects} <- 1 0.000 0.000 ImageFile.py:124(load)
{method 'seek' of 'cStringIO.StringO' objects} <- 1 0.000 0.000 HydrusFileHandling.py:64(GenerateThumbnail)
{method 'release' of 'thread.lock' objects} <- 1 0.000 0.000 threading.py:373(notify)
_util.py:4(isStringType) <- 1 0.000 0.000 JpegImagePlugin.py:560(_save)
{cStringIO.StringIO} <- 1 0.000 0.000 HydrusFileHandling.py:64(GenerateThumbnail)
ClientController.py:387(GetClientFilesManager) <- 1 0.000 0.000 ClientDB.py:4055(_ImportFile)
JpegImagePlugin.py:604(validate_qtables) <- 1 0.000 0.000 JpegImagePlugin.py:560(_save)
HydrusData.py:1678(GetKWArgs) <- 1 0.000 0.000 HydrusDB.py:180(_ProcessJob)
{method 'disable' of '_lsprof.Profiler' objects} <-
{method 'flush' of 'cStringIO.StringO' objects} <- 1 0.000 0.000 ImageFile.py:442(_save)
{method 'close' of 'cStringIO.StringO' objects} <- 1 0.000 0.000 HydrusFileHandling.py:64(GenerateThumbnail)
HydrusData.py:1676(GetArgs) <- 1 0.000 0.000 HydrusDB.py:180(_ProcessJob)
ImageFile.py:260(load_end) <- 1 0.000 0.000 ImageFile.py:124(load)
{method 'lower' of 'str' objects} <- 1 0.000 0.000 Image.py:1592(save)
{method 'cleanup' of 'ImagingEncoder' objects} <- 1 0.000 0.000 ImageFile.py:442(_save)</pre>
<p>Which is essentially the same data twice, in simple and expanded form.</p>
<p>It is very helpful to me to have a profile. You can generate one by going <i>help->debug->db profile mode</i>, which tells the client to generate profile information for every subsequent database request. This will spam your logfile, so don't leave it on for a very long time (you can turn it off by hitting the help menu entry again).</p>
<p>Turn on profile mode, do the thing that runs slow for you (importing a file, fetching some tags, whatever), and then shut the client down and go to your logfile, which should be at install_dir/logs/client.log. At the bottom of that file will be some big tables with timing information for different code calls. Find the ones labelled for your problem request (e.g. "import_file") and copy and paste that to me.</p>
<p><i>gui profile mode</i> is experimental and very log heavy. Feel free to play with it, but it is really for my own purposes. Almost everything that is slow in the program is due to my inefficient database queries.</p>
<p>You can <a href="contact.html">contact me</a> however you like--through email, on the 8chan board, on tumblr, github, whatever you prefer.</p>
</div>
</body>
</html>

21
help/running_from_source.html
View File

@ -9,31 +9,32 @@
<h3>running from source</h3>
<p>I write the client and server entirely in <a href="https://python.org">python</a>, which can run straight from source. I don't recommended this for someone who just wants to get the program working, but rather for those who have a general interest or wish to modify the program.</p>
<h3>what you will need</h3>
<p>You will need to install python 2.7 and several python modules. The best way to figure it out is just to keep running client.pyw and see what it complains about missing.</p>
<p>I recently got the client working on Ubuntu 14.04, which required:</p>
<p>You will need to install python 2.7 and a number of python modules. Most of it you can get through pip. I think this will do for most systems:</p>
<ul>
<li>(sudo) pip install beautifulsoup4 flvlib hsaudiotag lxml lz4 mp3play nose numpy pafy Pillow psutil pycrypto PyPDF2 PySocks python-potr PyYAML Send2Trash twisted</li>
</ul>
<p>Although you may want to do them one at a time, or in a different order. Your specific system may also need some of them from different sources and will need some complicated things installed separately. The best way to figure it out is just to keep running client.pyw and see what it complains about missing.</p>
<p>I use Ubuntu 14.04, which also requires something like:</p>
<p><ul>
<li>sudo apt-get install python2.7-dev</li>
<li>sudo pip install beautifulsoup4 pyyaml python-potr hsaudiotag PyPDF2 flvlib pafy lz4</li>
<li>sudo apt-get install python-numpy</li>
<li>sudo apt-get install python-opencv</li>
<li>sudo apt-get install python-wxversion</li>
<li>and installing a recent version of wxPython, which is difficult and best done <a href="http://wiki.wxpython.org/CheckInstall">like so</a></li>
<li>If you install wxPython as above and you get missing .so errors, try running 'sudo ldconfig'</li>
<li>If you install wxPython as above and you get missing .so errors, try running 'sudo ldconfig'.</li>
</ul></p>
<p>YMMV. Feel free to email me if you run into trouble or discover any neat tricks.</p>
<p>OS X 10.9 requires a similar pattern to above, though wx is simpler. I use the cocoa package <a href="http://wxpython.org/download.php#osx">straight from wxPython's site</a>. One user reports that this worked for 10.7:</p>
<p>OS X 10.9 is similar, though wx is simpler. I use the cocoa package <a href="http://wxpython.org/download.php#osx">straight from wxPython's site</a>. This should do the rest:</p>
<p><ul>
<li>sudo pip install beautifulsoup4 pyyaml python-potr hsaudiotag PyPDF2 twisted flvlib pafy numpy Pillow lxml lz4</li>
<li>brew tap homebrew/homebrew-science</li>
<li>brew install opencv</li>
<li>export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH</li>
</ul></p>
<p>For Windows, you can do the same thing with easy_install and pip and module installers. Since it doesn't natively come with Python, you'll probably need to go through a bigger list. You'll find <a href="http://www.lfd.uci.edu/~gohlke/pythonlibs/">this</a> page very helpful. I have a fair bit of experience with Windows python, so send me a mail if you need help.</a>
<p>For Windows, depending on which compiler you are using, pip can have problems building some modules like lz4 and lxml. <a href="http://www.lfd.uci.edu/~gohlke/pythonlibs/">This page</a> has a lot of prebuilt binaries--I have found it very helpful many times. wxPython is easy--just download from <a href="http://wxpython.org/download.php#osx">here</a>. I have a fair bit of experience with Windows python, so send me a mail if you need help.</a>
<p>A user has also created <a href="https://github.com/eider-i128/hydrus-win">an environment</a> to help Windows users run from source and build a distribution.</p>
<p>Some people have encountered problems with wxPython 3.0, so you might want to try 2.9.x. Again, YMMV.</p>
<p>You'll probably want to install Pillow (pip should do it for Linux/OS X, easy_install for Windows) instead of PIL, but go back to PIL if Pillow doesn't work.</p>
<p>If you want to import videos, you will need to put a static <a href="https://ffmpeg.org/">FFMPEG</a> executable in the install_dir/bin directory. Have a look at how I do it in the extractable compiled releases if you can't figure it out. You can either copy the exe from one of those releases, or download the latest build right from the FFMPEG site. I don't include these exes in the source release just because they are so big.</a>
<p>Once you have everything set up, client.pyw and server.pyw should look for and run off the database files just like the executables.</p>
<p>If you don't have ffmpeg in your path and you want to import videos, you will need to put a static <a href="https://ffmpeg.org/">FFMPEG</a> executable in the install_dir/bin directory. Have a look at how I do it in the extractable compiled releases if you can't figure it out. You can either copy the exe from one of those releases, or download the latest build right from the FFMPEG site. I don't include these exes in the source release just because they are so big.</a>
<p>Once you have everything set up, client.pyw and server.pyw should look for and run off client.db and server.db just like the executables.</p>
<p>I develop hydrus on 64-bit Win 7, so the program is much more stable and reasonable on Windows. I do not have as much experience with Linux or OS X, so I would particularly appreciate your Linux/OS X bug reports and any informed suggestions.</p>
<h3>my code</h3>
<p>My nerdism is INFJ, not INTP/J. My coding style is unusual, and everything is pretty much hacked together, but please do look through the source if you are interested in how things work and ask me if you don't understand something.</p>

15
include/ClientController.py
View File

@ -776,22 +776,25 @@ class Controller( HydrusController.HydrusController ):
wx.CallAfter( self.Exit )
while not self._db.LoopIsFinished(): time.sleep( 0.1 )
while not self._db.LoopIsFinished():
time.sleep( 0.1 )
self._db.RestoreBackup( path )
cmd = [ sys.executable ]
while not HydrusGlobals.shutdown_complete:
time.sleep( 0.1 )
cmd.extend( sys.argv )
subprocess.Popen( cmd )
HydrusData.RestartProcess()
restart_thread = threading.Thread( target = THREADRestart, name = 'Application Restart Thread' )
restart_thread.start()

50
include/ClientDB.py
View File

@ -3567,31 +3567,6 @@ class DB( HydrusDB.HydrusDB ):
return reason_id
def _GetRatingsMediaResult( self, service_key, min, max ):
service_id = self._GetServiceId( service_key )
half_point = ( min + max ) / 2
tighter_min = ( min + half_point ) / 2
tighter_max = ( max + half_point ) / 2
# I know this is horrible, ordering by random, but I can't think of a better way to do it right now
result = self._c.execute( 'SELECT hash_id FROM local_ratings, files_info USING ( hash_id ) WHERE local_ratings.service_id = ? AND files_info.service_id = ? AND rating BETWEEN ? AND ? ORDER BY RANDOM() LIMIT 1;', ( service_id, self._local_file_service_id, tighter_min, tighter_max ) ).fetchone()
if result is None: result = self._c.execute( 'SELECT hash_id FROM local_ratings, files_info USING ( hash_id ) WHERE local_ratings.service_id = ? AND files_info.service_id = ? AND rating BETWEEN ? AND ? ORDER BY RANDOM() LIMIT 1;', ( service_id, self._local_file_service_id, min, max ) ).fetchone()
if result is None: return None
else:
( hash_id, ) = result
( media_result, ) = self._GetMediaResults( CC.COMBINED_FILE_SERVICE_KEY, { hash_id } )
return media_result
def _GetService( self, service_id ):
if service_id in self._service_cache: service = self._service_cache[ service_id ]
@ -4986,7 +4961,6 @@ class DB( HydrusDB.HydrusDB ):
elif action == 'options': result = self._GetOptions( *args, **kwargs )
elif action == 'pending': result = self._GetPending( *args, **kwargs )
elif action == 'pixiv_account': result = self._GetYAMLDump( YAML_DUMP_ID_SINGLE, 'pixiv_account' )
elif action == 'ratings_media_result': result = self._GetRatingsMediaResult( *args, **kwargs )
elif action == 'remote_booru': result = self._GetYAMLDump( YAML_DUMP_ID_REMOTE_BOORU, *args, **kwargs )
elif action == 'remote_boorus': result = self._GetYAMLDump( YAML_DUMP_ID_REMOTE_BOORU )
elif action == 'service_info': result = self._GetServiceInfo( *args, **kwargs )
@ -5136,7 +5110,17 @@ class DB( HydrusDB.HydrusDB ):
( dump_type, dump_name, version, serialisable_info ) = obj.GetSerialisableTuple()
dump = json.dumps( serialisable_info )
try:
dump = json.dumps( serialisable_info )
except:
HydrusData.Print( obj )
HydrusData.Print( serialisable_info )
raise Exception( 'Trying to json dump the object ' + HydrusData.ToUnicode( obj ) + ' with name ' + dump_name + ' caused an error. Its serialisable info has been dumped to the log.' )
self._c.execute( 'DELETE FROM json_dumps_named WHERE dump_type = ? AND dump_name = ?;', ( dump_type, dump_name ) )
@ -5146,7 +5130,17 @@ class DB( HydrusDB.HydrusDB ):
( dump_type, version, serialisable_info ) = obj.GetSerialisableTuple()
dump = json.dumps( serialisable_info )
try:
dump = json.dumps( serialisable_info )
except:
HydrusData.Print( obj )
HydrusData.Print( serialisable_info )
raise Exception( 'Trying to json dump the object ' + HydrusData.ToUnicode( obj ) + ' caused an error. Its serialisable info has been dumped to the log.' )
self._c.execute( 'DELETE FROM json_dumps WHERE dump_type = ?;', ( dump_type, ) )

29
include/ClientDownloading.py
View File

@ -8,6 +8,7 @@ import HydrusPaths
import HydrusSerialisable
import HydrusThreading
import json
import lxml # to force import for later bs4 stuff
import os
import pafy
import re
@ -195,6 +196,10 @@ def GetImageboardThreadURLs( thread_url ):
return ( json_url, file_base )
def GetSoup( html ):
return bs4.BeautifulSoup( html, 'lxml' )
def GetYoutubeFormats( youtube_url ):
try: p = pafy.Pafy( youtube_url )
@ -258,7 +263,7 @@ def THREADDownloadURL( job_key, url, url_string ):
def Parse4chanPostScreen( html ):
soup = bs4.BeautifulSoup( html )
soup = GetSoup( html )
title_tag = soup.find( 'title' )
@ -302,7 +307,7 @@ def Parse4chanPostScreen( html ):
def ParsePageForURLs( html, starting_url ):
soup = bs4.BeautifulSoup( html )
soup = GetSoup( html )
all_links = soup.find_all( 'a' )
@ -532,7 +537,7 @@ class GalleryBooru( Gallery ):
urls_set = set()
urls = []
soup = bs4.BeautifulSoup( html )
soup = GetSoup( html )
# this catches 'post-preview' along with 'post-preview not-approved' sort of bullshit
def starts_with_classname( classname ): return classname is not None and classname.startswith( self._thumb_classname )
@ -590,7 +595,7 @@ class GalleryBooru( Gallery ):
( search_url, search_separator, advance_by_page_num, thumb_classname, image_id, image_data, tag_classnames_to_namespaces ) = self._booru.GetData()
soup = bs4.BeautifulSoup( html )
soup = GetSoup( html )
image_base = None
@ -788,7 +793,7 @@ class GalleryDeviantArt( Gallery ):
urls = []
soup = bs4.BeautifulSoup( html )
soup = GetSoup( html )
thumbs_container = soup.find( class_ = 'zones-container' )
@ -828,7 +833,7 @@ class GalleryDeviantArt( Gallery ):
def _ParseImagePage( self, html, referer_url ):
soup = bs4.BeautifulSoup( html )
soup = GetSoup( html )
download_button = soup.find( 'a', class_ = 'dev-page-download' )
@ -988,7 +993,7 @@ class GalleryHentaiFoundry( Gallery ):
urls_set = set()
soup = bs4.BeautifulSoup( html )
soup = GetSoup( html )
def correct_url( href ):
@ -1057,7 +1062,7 @@ class GalleryHentaiFoundry( Gallery ):
raise Exception( 'Could not parse image url!' + os.linesep + HydrusData.ToUnicode( e ) )
soup = bs4.BeautifulSoup( html )
soup = GetSoup( html )
tags = []
@ -1155,7 +1160,7 @@ class GalleryNewgrounds( Gallery ):
def _ParseGalleryPage( self, html, url_base ):
soup = bs4.BeautifulSoup( html )
soup = GetSoup( html )
fatcol = soup.find( 'div', class_ = 'fatcol' )
@ -1191,7 +1196,7 @@ class GalleryNewgrounds( Gallery ):
def _ParseImagePage( self, html, url_base ):
soup = bs4.BeautifulSoup( html )
soup = GetSoup( html )
tags = set()
@ -1309,7 +1314,7 @@ class GalleryPixiv( Gallery ):
urls = []
soup = bs4.BeautifulSoup( html )
soup = GetSoup( html )
manga_links = soup.find_all( class_ = 'manga' )
thumbnail_links = soup.find_all( class_ = 'work' )
@ -1348,7 +1353,7 @@ class GalleryPixiv( Gallery ):
raise Exception( page_url + ' was manga, not a single image, so could not be downloaded.' )
soup = bs4.BeautifulSoup( html )
soup = GetSoup( html )
#

62
include/ClientGUI.py
View File

@ -592,6 +592,42 @@ class FrameGUI( ClientGUICommon.FrameThatResizes ):
wx.CallAfter( gc.collect )
def _Exit( self, restart = False ):
if HC.options[ 'confirm_client_exit' ]:
if restart:
text = 'Are you sure you want to restart the client? (Will auto-yes in 15 seconds)'
else:
text = 'Are you sure you want to exit the client? (Will auto-yes in 15 seconds)'
with ClientGUIDialogs.DialogYesNo( self, text ) as dlg:
call_later = wx.CallLater( 15000, dlg.EndModal, wx.ID_YES )
if dlg.ShowModal() == wx.ID_NO:
call_later.Stop()
return
call_later.Stop()
if restart:
HydrusGlobals.restart = True
self._controller.Exit()
def _FetchIP( self, service_key ):
with ClientGUIDialogs.DialogTextEntry( self, 'Enter the file\'s hash.' ) as dlg:
@ -642,6 +678,8 @@ class FrameGUI( ClientGUICommon.FrameThatResizes ):
menu.AppendSeparator()
menu.Append( ClientCaches.MENU_EVENT_ID_TO_ACTION_CACHE.GetPermanentId( 'options' ), p( '&Options' ) )
menu.AppendSeparator()
if not HC.PLATFORM_LINUX:
menu.Append( ClientCaches.MENU_EVENT_ID_TO_ACTION_CACHE.GetPermanentId( 'restart' ), p( '&Restart' ) )
menu.Append( ClientCaches.MENU_EVENT_ID_TO_ACTION_CACHE.GetPermanentId( 'exit' ), p( '&Exit' ) )
return ( menu, p( '&File' ), True )
@ -2023,26 +2061,7 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
def EventExit( self, event ):
if HC.options[ 'confirm_client_exit' ]:
text = 'Are you sure you want to exit the client? (Will auto-yes in 15 seconds)'
with ClientGUIDialogs.DialogYesNo( self, text ) as dlg:
call_later = wx.CallLater( 15000, dlg.EndModal, wx.ID_YES )
if dlg.ShowModal() == wx.ID_NO:
call_later.Stop()
return
call_later.Stop()
self._controller.Exit()
self._Exit()
def EventFocus( self, event ):
@ -2118,7 +2137,7 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
elif command == 'delete_orphans': self._DeleteOrphans()
elif command == 'delete_pending': self._DeletePending( data )
elif command == 'delete_service_info': self._DeleteServiceInfo()
elif command == 'exit': self.EventExit( event )
elif command == 'exit': self._Exit()
elif command == 'fetch_ip': self._FetchIP( data )
elif command == 'force_idle':
@ -2191,6 +2210,7 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
if page is not None: page.RefreshQuery()
elif command == 'regenerate_thumbnails': self._RegenerateThumbnails()
elif command == 'restart': self._Exit( restart = True )
elif command == 'restore_database': self._controller.RestoreDatabase()
elif command == 'review_services': self._ReviewServices()
elif command == 'save_gui_session': self._SaveGUISession()

37
include/ClientGUICanvas.py
View File

@ -246,7 +246,7 @@ class Animation( wx.Window ):
etype = event.GetEventType()
if not ( event.ShiftDown() or event.ControlDown() or event.CmdDown() or event.AltDown() ):
if not ( event.ShiftDown() or event.CmdDown() or event.AltDown() ):
if etype == wx.wxEVT_LEFT_DOWN:
@ -649,6 +649,7 @@ class Canvas( object ):
self._canvas_zoom = 1.0
self._last_drag_coordinates = None
self._last_motion_coordinates = ( 0, 0 )
self._total_drag_delta = ( 0, 0 )
self.SetBackgroundColour( wx.Colour( *HC.options[ 'gui_colours' ][ 'media_background' ] ) )
@ -1844,16 +1845,35 @@ class CanvasFullscreenMediaList( ClientMedia.ListeningMediaList, CanvasWithDetai
self._focus_holder.SetFocus()
( x, y ) = event.GetPosition()
show_mouse = False
if ( x, y ) != self._last_motion_coordinates:
self._last_motion_coordinates = ( x, y )
show_mouse = True
if event.Dragging() and self._last_drag_coordinates is not None:
( old_x, old_y ) = self._last_drag_coordinates
( x, y ) = event.GetPosition()
( delta_x, delta_y ) = ( x - old_x, y - old_y )
if HC.PLATFORM_WINDOWS: self.WarpPointer( old_x, old_y )
else: self._last_drag_coordinates = ( x, y )
if HC.PLATFORM_WINDOWS:
show_mouse = False
self.WarpPointer( old_x, old_y )
else:
show_mouse = True
self._last_drag_coordinates = ( x, y )
( old_delta_x, old_delta_y ) = self._total_drag_delta
@ -1862,9 +1882,12 @@ class CanvasFullscreenMediaList( ClientMedia.ListeningMediaList, CanvasWithDetai
self._DrawCurrentMedia()
self.SetCursor( wx.StockCursor( wx.CURSOR_ARROW ) )
if show_mouse:
self._timer_cursor_hide.Start( 800, wx.TIMER_ONE_SHOT )
self.SetCursor( wx.StockCursor( wx.CURSOR_ARROW ) )
self._timer_cursor_hide.Start( 800, wx.TIMER_ONE_SHOT )
def EventDragBegin( self, event ):

59
include/ClientGUICommon.py
View File

@ -396,47 +396,44 @@ class AutoCompleteDropdown( wx.Panel ):
HydrusGlobals.client_controller.ResetIdleTimer()
if event.KeyCode in ( wx.WXK_TAB, wx.WXK_NUMPAD_TAB ):
if event.KeyCode in ( wx.WXK_INSERT, wx.WXK_NUMPAD_INSERT ):
if event.ShiftDown():
if self._intercept_key_events:
if self._intercept_key_events:
self._intercept_key_events = False
( r, g, b ) = HC.options[ 'gui_colours' ][ 'autocomplete_background' ]
if r != g or r != b or g != b:
self._intercept_key_events = False
colour = wx.Colour( g, b, r )
( r, g, b ) = HC.options[ 'gui_colours' ][ 'autocomplete_background' ]
elif r > 127:
if r != g or r != b or g != b:
colour = wx.Colour( g, b, r )
elif r > 127:
colour = wx.Colour( g, b, r / 2 )
else:
colour = wx.Colour( g, b, r * 2 )
colour = wx.Colour( g, b, r / 2 )
else:
self._intercept_key_events = True
colour = wx.Colour( g, b, r * 2 )
colour = wx.Colour( *HC.options[ 'gui_colours' ][ 'autocomplete_background' ] )
self._text_ctrl.SetBackgroundColour( colour )
self._text_ctrl.Refresh()
else:
self._UpdateList()
self._intercept_key_events = True
self._lag_timer.Stop()
colour = wx.Colour( *HC.options[ 'gui_colours' ][ 'autocomplete_background' ] )
self._text_ctrl.SetBackgroundColour( colour )
self._text_ctrl.Refresh()
elif event.KeyCode == wx.WXK_SPACE and event.RawControlDown(): # this is control, not command on os x, for which command+space does some os stuff
self._UpdateList()
self._lag_timer.Stop()
elif self._intercept_key_events:
if event.KeyCode in ( wx.WXK_RETURN, wx.WXK_NUMPAD_ENTER ) and self._ShouldTakeResponsibilityForEnter():
@ -2535,7 +2532,7 @@ class ListBox( wx.ScrolledWindow ):
def EventKeyDown( self, event ):
shift = event.ShiftDown()
ctrl = event.CmdDown() or event.ControlDown()
ctrl = event.CmdDown()
key_code = event.GetKeyCode()
@ -2606,7 +2603,7 @@ class ListBox( wx.ScrolledWindow ):
hit_index = self._GetIndexUnderMouse( event )
shift = event.ShiftDown()
ctrl = event.CmdDown() or event.ControlDown()
ctrl = event.CmdDown()
self._Hit( shift, ctrl, hit_index )
@ -2823,7 +2820,7 @@ class ListBoxTags( ListBox ):
hit_index = self._GetIndexUnderMouse( event )
shift = event.ShiftDown()
ctrl = event.CmdDown() or event.ControlDown()
ctrl = event.CmdDown()
self._Hit( shift, ctrl, hit_index )
@ -2835,7 +2832,7 @@ class ListBoxTags( ListBox ):
hit_index = self._GetIndexUnderMouse( event )
shift = event.ShiftDown()
ctrl = event.CmdDown() or event.ControlDown()
ctrl = event.CmdDown()
self._Hit( shift, ctrl, hit_index )
@ -3132,7 +3129,7 @@ class ListBoxTagsAutocompleteDropdown( ListBoxTags ):
shift = event.ShiftDown()
ctrl = event.CmdDown() or event.ControlDown()
ctrl = event.CmdDown()
self._Hit( shift, ctrl, hit_index )

75
include/ClientGUIDialogs.py
View File

@ -4140,48 +4140,71 @@ class DialogSetupExport( Dialog ):
directory = self._directory_picker.GetPath()
if not os.path.exists( directory ):
os.makedirs( directory )
pattern = self._pattern.GetValue()
terms = ClientFiles.ParseExportPhrase( pattern )
client_files_manager = HydrusGlobals.client_controller.GetClientFilesManager()
for ( ( ordering_index, media ), mime, path ) in self._paths.GetClientData():
self._export.Disable()
to_do = self._paths.GetClientData()
num_to_do = len( to_do )
def do_it():
try:
for ( index, ( ( ordering_index, media ), mime, path ) ) in enumerate( to_do ):
hash = media.GetHash()
if export_tag_txts:
try:
tags_manager = media.GetTagsManager()
wx.CallAfter( self._export.SetLabel, HydrusData.ConvertValueRangeToPrettyString( index + 1, num_to_do ) )
tags = tags_manager.GetCurrent()
hash = media.GetHash()
filename = ClientFiles.GenerateExportFilename( media, terms )
txt_path = os.path.join( directory, filename + '.txt' )
with open( txt_path, 'wb' ) as f:
if export_tag_txts:
f.write( HydrusData.ToByteString( os.linesep.join( tags ) ) )
tags_manager = media.GetTagsManager()
tags = tags_manager.GetCurrent()
filename = ClientFiles.GenerateExportFilename( media, terms )
txt_path = os.path.join( directory, filename + '.txt' )
with open( txt_path, 'wb' ) as f:
f.write( HydrusData.ToByteString( os.linesep.join( tags ) ) )
source_path = client_files_manager.GetFilePath( hash, mime )
shutil.copy2( source_path, path )
try: os.chmod( path, stat.S_IWRITE | stat.S_IREAD )
except: pass
except:
wx.MessageBox( 'Encountered a problem while attempting to export file with index ' + str( ordering_index + 1 ) + ':' + os.linesep * 2 + traceback.format_exc() )
break
source_path = client_files_manager.GetFilePath( hash, mime )
shutil.copy2( source_path, path )
try: os.chmod( path, stat.S_IWRITE | stat.S_IREAD )
except: pass
except:
wx.CallAfter( wx.MessageBox, 'Encountered a problem while attempting to export file with index ' + str( ordering_index + 1 ) + ':' + os.linesep * 2 + traceback.format_exc() )
break
wx.CallAfter( self._export.SetLabel, 'done!' )
time.sleep( 1 )
wx.CallAfter( self._export.SetLabel, 'export' )
wx.CallAfter( self._export.Enable )
HydrusGlobals.client_controller.CallToThread( do_it )
def EventOpenLocation( self, event ):

92
include/ClientGUIDialogsManage.py
View File

@ -4710,7 +4710,7 @@ class DialogManageOptions( ClientGUIDialogs.Dialog ):
vbox.AddF( gridbox, CC.FLAGS_EXPAND_PERPENDICULAR )
text = 'If you want to disable automatic autocomplete results fetching, use the Tab key to fetch results manually.'
text = 'If you want to disable automatic autocomplete results fetching, use Ctrl+Space to fetch results manually.'
vbox.AddF( wx.StaticText( self, label = text ), CC.FLAGS_EXPAND_PERPENDICULAR )
@ -7842,7 +7842,14 @@ class DialogManageTagParents( ClientGUIDialogs.Dialog ):
if self._account.HasPermission( HC.RESOLVE_PETITIONS ): reason = 'admin'
else:
pair_strings = os.linesep.join( ( child + '->' + parent for ( child, parent ) in new_pairs ) )
if len( new_pairs ) > 10:
pair_strings = 'The many pairs you entered.'
else:
pair_strings = os.linesep.join( ( child + '->' + parent for ( child, parent ) in new_pairs ) )
message = 'Enter a reason for:' + os.linesep * 2 + pair_strings + os.linesep * 2 + 'To be added. A janitor will review your petition.'
@ -7877,7 +7884,15 @@ class DialogManageTagParents( ClientGUIDialogs.Dialog ):
if self._service_key != CC.LOCAL_TAG_SERVICE_KEY:
pair_strings = os.linesep.join( ( child + '->' + parent for ( child, parent ) in current_pairs ) )
if len( current_pairs ) > 10:
pair_strings = 'The many pairs you entered.'
else:
pair_strings = os.linesep.join( ( child + '->' + parent for ( child, parent ) in current_pairs ) )
if len( current_pairs ) > 1: message = 'The pairs:' + os.linesep * 2 + pair_strings + os.linesep * 2 + 'Already exist.'
else: message = 'The pair ' + pair_strings + ' already exists.'
@ -7906,6 +7921,11 @@ class DialogManageTagParents( ClientGUIDialogs.Dialog ):
for pair in current_pairs: self._pairs_to_reasons[ pair ] = reason
else:
do_it = False
@ -7918,8 +7938,15 @@ class DialogManageTagParents( ClientGUIDialogs.Dialog ):
if len( pending_pairs ) > 0:
pair_strings = os.linesep.join( ( child + '->' + parent for ( child, parent ) in pending_pairs ) )
if len( pending_pairs ) > 10:
pair_strings = 'The many pairs you entered.'
else:
pair_strings = os.linesep.join( ( child + '->' + parent for ( child, parent ) in pending_pairs ) )
if len( pending_pairs ) > 1: message = 'The pairs:' + os.linesep * 2 + pair_strings + os.linesep * 2 + 'Are pending.'
else: message = 'The pair ' + pair_strings + ' is pending.'
@ -7936,8 +7963,15 @@ class DialogManageTagParents( ClientGUIDialogs.Dialog ):
if len( petitioned_pairs ) > 0:
pair_strings = ', '.join( ( child + '->' + parent for ( child, parent ) in petitioned_pairs ) )
if len( petitioned_pairs ) > 10:
pair_strings = 'The many pairs you entered.'
else:
pair_strings = os.linesep.join( ( child + '->' + parent for ( child, parent ) in petitioned_pairs ) )
if len( petitioned_pairs ) > 1: message = 'The pairs:' + os.linesep * 2 + pair_strings + os.linesep * 2 + 'Are petitioned.'
else: message = 'The pair ' + pair_strings + ' is petitioned.'
@ -8407,7 +8441,14 @@ class DialogManageTagSiblings( ClientGUIDialogs.Dialog ):
if self._account.HasPermission( HC.RESOLVE_PETITIONS ): reason = 'admin'
else:
pair_strings = os.linesep.join( ( old + '->' + new for ( old, new ) in new_pairs ) )
if len( new_pairs ) > 10:
pair_strings = 'The many pairs you entered.'
else:
pair_strings = os.linesep.join( ( old + '->' + new for ( old, new ) in new_pairs ) )
message = 'Enter a reason for:' + os.linesep * 2 + pair_strings + os.linesep * 2 + 'To be added. A janitor will review your petition.'
@ -8442,7 +8483,14 @@ class DialogManageTagSiblings( ClientGUIDialogs.Dialog ):
if self._service_key != CC.LOCAL_TAG_SERVICE_KEY:
pair_strings = os.linesep.join( ( old + '->' + new for ( old, new ) in current_pairs ) )
if len( current_pairs ) > 10:
pair_strings = 'The many pairs you entered.'
else:
pair_strings = os.linesep.join( ( old + '->' + new for ( old, new ) in current_pairs ) )
if len( current_pairs ) > 1: message = 'The pairs:' + os.linesep * 2 + pair_strings + os.linesep * 2 + 'Already exist.'
else: message = 'The pair ' + pair_strings + ' already exists.'
@ -8468,6 +8516,10 @@ class DialogManageTagSiblings( ClientGUIDialogs.Dialog ):
for pair in current_pairs: self._pairs_to_reasons[ pair ] = reason
else:
do_it = False
@ -8481,8 +8533,15 @@ class DialogManageTagSiblings( ClientGUIDialogs.Dialog ):
if len( pending_pairs ) > 0:
pair_strings = os.linesep.join( ( old + '->' + new for ( old, new ) in pending_pairs ) )
if len( pending_pairs ) > 10:
pair_strings = 'The many pairs you entered.'
else:
pair_strings = os.linesep.join( ( old + '->' + new for ( old, new ) in pending_pairs ) )
if len( pending_pairs ) > 1: message = 'The pairs:' + os.linesep * 2 + pair_strings + os.linesep * 2 + 'Are pending.'
else: message = 'The pair ' + pair_strings + ' is pending.'
@ -8499,8 +8558,15 @@ class DialogManageTagSiblings( ClientGUIDialogs.Dialog ):
if len( petitioned_pairs ) > 0:
pair_strings = ', '.join( ( old + '->' + new for ( old, new ) in petitioned_pairs ) )
if len( petitioned_pairs ) > 10:
pair_strings = 'The many pairs you entered.'
else:
pair_strings = ', '.join( ( old + '->' + new for ( old, new ) in petitioned_pairs ) )
if len( petitioned_pairs ) > 1: message = 'The pairs:' + os.linesep * 2 + pair_strings + os.linesep * 2 + 'Are petitioned.'
else: message = 'The pair ' + pair_strings + ' is petitioned.'

4
include/ClientGUIMedia.py
View File

@ -1453,8 +1453,8 @@ class MediaPanelThumbnails( MediaPanel ):
self._dirty_canvas_pages = []
self.Refresh()
self.Refresh()

8
include/ClientImporting.py
View File

@ -412,7 +412,7 @@ class GalleryImport( HydrusSerialisable.SerialisableBase ):
self._WorkOnFiles( page_key )
time.sleep( 1 )
time.sleep( 0.1 )
HydrusGlobals.client_controller.WaitUntilPubSubsEmpty()
@ -1289,7 +1289,7 @@ class PageOfImagesImport( HydrusSerialisable.SerialisableBase ):
html = HydrusGlobals.client_controller.DoHTTP( HC.GET, page_url )
soup = bs4.BeautifulSoup( html )
soup = ClientDownloading.GetSoup( html )
#
@ -1401,7 +1401,7 @@ class PageOfImagesImport( HydrusSerialisable.SerialisableBase ):
self._WorkOnFiles( page_key )
time.sleep( 1 )
time.sleep( 0.1 )
HydrusGlobals.client_controller.WaitUntilPubSubsEmpty()
@ -2645,7 +2645,7 @@ class ThreadWatcherImport( HydrusSerialisable.SerialisableBase ):
self._WorkOnFiles( page_key )
time.sleep( 1 )
time.sleep( 0.1 )
HydrusGlobals.client_controller.WaitUntilPubSubsEmpty()

2
include/HydrusConstants.py
View File

@ -53,7 +53,7 @@ options = {}
# Misc
NETWORK_VERSION = 17
SOFTWARE_VERSION = 185
SOFTWARE_VERSION = 186
UNSCALED_THUMBNAIL_DIMENSIONS = ( 200, 200 )

6
include/HydrusData.py
View File

@ -853,6 +853,12 @@ def RecordRunningStart( instance ):
f.write( ToByteString( record_string ) )
def RestartProcess():
time.sleep( 1 ) # time for ports to unmap
os.execl( sys.executable, sys.executable, *sys.argv )
def ShowExceptionDefault( e ):
if isinstance( e, HydrusExceptions.ShutdownException ):

3
include/HydrusGlobals.py
View File

@ -11,3 +11,6 @@ is_db_updated = False
db_profile_mode = False
pubsub_profile_mode = False
server_busy = False
shutdown_complete = False
restart = False

8
include/TestDB.py
View File

@ -955,14 +955,6 @@ class TestClientDB( unittest.TestCase ):
self.assertTrue( result, ( pixiv_id, password ) )
def test_ratings_media_result( self ):
# add some ratings, slice different media results?
# check exactly what this does again, figure a good test
pass
def test_repo_downloads( self ):
result = self._read( 'downloads' )

30
include/TestFunctions.py
View File

@ -51,4 +51,34 @@ class TestClientDownloadingFunctions( unittest.TestCase ):
self.assertEqual( i_pretty, '123,456,789' )
def test_unicode_conversion( self ):
u = u'here is a unicode character: \u76f4'
b = HydrusData.ToByteString( u )
bu = HydrusData.ToUnicode( b )
self.assertEqual( type( b ), str )
self.assertEqual( b, 'here is a unicode character: \xe7\x9b\xb4' )
self.assertEqual( type( bu ), unicode )
self.assertEqual( bu, u )
d = {}
u = HydrusData.ToUnicode( d )
b = HydrusData.ToByteString( d )
self.assertEqual( type( u ), unicode )
self.assertEqual( u, u'{}' )
self.assertEqual( type( b ), str )
self.assertEqual( b, '{}' )
pretty_num = HydrusData.ConvertIntToPrettyString( 123456789 )
self.assertEqual( type( pretty_num ), unicode )

2
server.py
View File

@ -98,7 +98,7 @@ except:
import traceback
HydrusData.Print( 'Critical error occured! Details written to crash.log!' )
print( 'Critical error occured! Details written to crash.log!' )
with open( 'crash.log', 'wb' ) as f: f.write( traceback.format_exc() )