RepoMirrors
/
hydrus
mirror of https://github.com/hydrusnetwork/hydrus.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Version 63

This commit is contained in:
Hydrus 2013-03-27 15:02:51 -05:00
parent 0d96611cff
commit e2cd61f733
17 changed files with 1209 additions and 785 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
client.pyw
View File

@ -4,6 +4,16 @@
# To Public License, Version 2, as published by Sam Hocevar. See
# http://sam.zoy.org/wtfpl/COPYING for more details.
import string
string.whitespace
# this is some woo woo - if you call it after the locale, it has 0xa0 (non-breaking space) (non-ascii!!) included
# if you call it before, the locale call doesn't update
# what a mess!
import locale
print( locale.setlocale( locale.LC_ALL, '' ) )
import os
from include import HydrusConstants as HC
from include import ClientController

144
help/faq.html
View File

@ -1,72 +1,74 @@
<html>
<head>
<title>faq</title>
<link href="hydrus.ico" rel="shortcut icon" />
<link href="style.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div class="content">
<a name="repositories"><h3>hold up, what is a repository?</h3></a>
<p>A <i>repository</i> is a special kind of server in the hydrus network that stores a certain kind of information—files or tag mappings, for instance—as submitted by users all over the internet. Those users periodically synchronise with the repository so they know what it stores. Hydrus network clients never send queries to repositories; they download and cache <i>all</i> of a repository's searchable metadata and perform queries over that cache, locally, on the client's computer.</p>
<a name="tags"><h3>hold up, what is a tag?</h3></a>
<p><a href="http://en.wikipedia.org/wiki/Tag_(metadata)">wiki</a></p>
<p>A <i>tag</i> is a small bit of text describing a single property of something. They make searching easy. Good examples are "flower" or "nicolas cage" or "the sopranos" or "2003". By combining several tags together ( e.g. [ 'tiger woods', 'sports illustrated', '2008' ] or [ 'cosplay', 'the legend of zelda' ] ), a huge image collection is reduced to a tiny and easy-to-digest sample, usually in less than a second.</p>
<p>A good word for the connection of a particular tag to a particular file is <i>mapping</i>.</p>
<p>In the hydrus network, all tags are automatically converted to lower case. 'Sunset Drive' becomes 'sunset drive'. Why?</p>
<ol>
<li>Although it may at first seem preferable to have proper capitalised titles, like 'The Lord of the Rings' rather than 'the lord of the rings', there are many, many special cases where style guides differ. There is no definitive correct capitalisation schema, so the simplest compromise is to not have any.</li>
<li>Searches become far easier when case is not matched. And when case does not matter, what point is there in recording it?</li>
</ol>
<p>Secondly, leading and trailing whitespace is removed, and multiple whitespace is collapsed to a single character. <pre>' yellow dress '</pre> becomes <pre>'yellow dress'</pre></p>
<p><a href="asolutionthatmaximisesutility.gif">Does this unjust censorship frustrate you?</a></p>
<a name="filenames"><h3>why not use filenames and folders?</h3></a>
<p>As a retrieval method, filenames and folders become worse and worse as the number of files increases. Why?</p>
<ul>
<li>A filename is not unique; did you mean this "04.jpg" or <i>this</i> "04.jpg"? Perhaps "04 (3).jpg"?</li>
<li>A filename is not guaranteed to describe the file correctly, nor is it proofed against trolling, e.g. hello.jpg</li>
<li>A filename is not guaranteed to stay the same, meaning other programs cannot rely on the filename address being valid or even returning the same data every time. This is the cause of a ton of behind-the-scenes and often redundant reindexing that slows nearly all other media management programs.</li>
<li>A filename is often—for <i>ridiculous</i> reasons—limited to a certain prohibitive character set; even when utf-8 is supported, some arbitrary ascii characters are usually not, and different localisations, operating systems and formatting conventions only make it worse.</p>
<li>Folders can offer context, but they are clunky and time-consuming to change. If you put each chapter of a comic in a different folder, for instance, reading several volumes in one sitting can be a pain. Nesting many folders adds navigation-latency and tends to induce less informative "04.jpg"-type filenames.</li>
</ul>
<p>So, the client tracks files by their <i>hash</i>.</p>
<p><i>BTW: when exporting files, the client names them by their hexadecimalised hash, like so: f099b5823f4e36a4bd6562812582f60e49e818cf445902b504b5533c6a5dad94.jpg. This will probably change to tag-munged in future.</i></p>
<p>Please do not tag your files with their exact original 'filename.jpg' on my public tag repo. <a href="http://www.youtube.com/watch?v=_yYS0ZZdsnA">Shed the concept of filenames as you would chains.</a></p>
<a name="hashes"><h3>hold up, what is a hash?</h3></a>
<p><a href="http://en.wikipedia.org/wiki/Hash_function">wiki</a></p>
<p>Hashes are a subject one usually has to be a software engineer to find interesting. If you don't care to digest the wiki page, the simple answer is that hashes are guaranteed unique names for things. It can be proven that f099b5823f4e36a4bd6562812582f60e49e818cf445902b504b5533c6a5dad94 refers to one particular file and no other. Hashes make excellent—if ugly—identifiers. In the client's normal operation, you will never encounter a file's hash; if you like a thumbnail, double-click it; the software handles the mathematics.</p>
<p><i>For those who </i>are<i> interested: hydrus uses SHA-256, which spits out 32-byte (256-bit) hashes. The software stores and searches over the hash densely, as 32 bytes, only encoding it to 64 hex characters when the user views it or copies to clipboard. SHA-256 is not perfect, but it is a great compromise candidate; it is secure for now, it is reasonably fast, it is available for most programming languages, and newer CPUs perform it more efficiently all the time. Maybe when NIST decides on the SHA-3 winner we will have a grand switch over.</i></p>
<a name="access_keys"><h3>hold up, what is an access key?</h3></a>
<p>The hydrus network's repositories do not use username/password, but instead a single combination identifier-password like this: <i>7ce4dbf18f7af8b420ee942bae42030aab344e91dc0e839260fcd71a4c9879e3</i></p>
<p>These hex numbers give you access to a particular account on a particular repository. They are long enough to be impossible to guess, and also randomly generated, so they reveal nothing personally identifying about you. Many people can use the same access key (and hence the same account) on a repository without consequence, although they will have to share bandwidth limits, and if one person screws around and gets the account banned, they will all lose access.</p>
<a name="other_platforms"><h3>why shouldn't I use a more mature platform?</h3></a>
<p>Some applications like ACDSee try to make finding files easier than browsing explorer, but they are all-too-often:</p>
<p>
<ul>
<li>weighed down by noob-friendly 'features'</li>
<li>lacking any easy and open method of sharing files or tags</li>
<li>proprietary, untrustworthy, and expensive</li>
<li>good at one specific job, but only that</li>
<li>cluttered with inefficient database code, and perpetual reindexing</li>
<li>designed by people who think the internet is AOL.com</li>
</ul>
</p>
<p>Some websites like flickr and danbooru have crowd-sourced tags and offer fairly effective retrieval, but then <i>they</i> all-too-often have:</p>
<p>
<ul>
<li>choked bandwidth/server CPU</li>
<li>high search latency</li>
<li>degraded image quality</li>
<li>small results sets (e.g. 12 results to a page)</li>
<li>arbitrary obscenity rules</li>
<li>intrusive advertisement</li>
<li>unreliable access</li>
<li>uncertain privacy</li>
</ul>
</p>
<p>The hydrus network attempts to combine the privacy and low latency of local searching with the efficiency of crowd-sourcing.</p>
<a name="delays"><h3>why can my friend not see what I just uploaded?</h3></a>
<p>The repositories do not work like conventional search engines; it takes a short but predictable while for changes to propagate to other users.</p>
<p>Remember that the client's searches only ever happen over its local cache of what is on the repository. Those caches are updated about once a day, so any changes you make will be delayed for others until their next update occurs. At the moment, the update period is 100,000 seconds, which is about 1 day and 4 hours.</p>
</div>
</body>
<html>
<head>
<title>faq</title>
<link href="hydrus.ico" rel="shortcut icon" />
<link href="style.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div class="content">
<a name="repositories"><h3>hold up, what is a repository?</h3></a>
<p>A <i>repository</i> is a special kind of server in the hydrus network that stores a certain kind of information—files or tag mappings, for instance—as submitted by users all over the internet. Those users periodically synchronise with the repository so they know what it stores. Hydrus network clients never send queries to repositories; they download and cache <i>all</i> of a repository's searchable metadata and perform queries over that cache, locally, on the client's computer.</p>
<a name="tags"><h3>hold up, what is a tag?</h3></a>
<p><a href="http://en.wikipedia.org/wiki/Tag_(metadata)">wiki</a></p>
<p>A <i>tag</i> is a small bit of text describing a single property of something. They make searching easy. Good examples are "flower" or "nicolas cage" or "the sopranos" or "2003". By combining several tags together ( e.g. [ 'tiger woods', 'sports illustrated', '2008' ] or [ 'cosplay', 'the legend of zelda' ] ), a huge image collection is reduced to a tiny and easy-to-digest sample, usually in less than a second.</p>
<p>A good word for the connection of a particular tag to a particular file is <i>mapping</i>.</p>
<p>In the hydrus network, all tags are automatically converted to lower case. 'Sunset Drive' becomes 'sunset drive'. Why?</p>
<ol>
<li>Although it may at first seem preferable to have proper capitalised titles, like 'The Lord of the Rings' rather than 'the lord of the rings', there are many, many special cases where style guides differ. There is no definitive correct capitalisation schema, so the simplest compromise is to not have any.</li>
<li>Searches become far easier when case is not matched. And when case does not matter, what point is there in recording it?</li>
</ol>
<p>Secondly, leading and trailing whitespace is removed, and multiple whitespace is collapsed to a single character. <pre>' yellow dress '</pre> becomes <pre>'yellow dress'</pre></p>
<p><a href="asolutionthatmaximisesutility.gif">Does this unjust censorship frustrate you?</a></p>
<a name="filenames"><h3>why not use filenames and folders?</h3></a>
<p>As a retrieval method, filenames and folders become worse and worse as the number of files increases. Why?</p>
<ul>
<li>A filename is not unique; did you mean this "04.jpg" or <i>this</i> "04.jpg"? Perhaps "04 (3).jpg"?</li>
<li>A filename is not guaranteed to describe the file correctly, nor is it proofed against trolling, e.g. hello.jpg</li>
<li>A filename is not guaranteed to stay the same, meaning other programs cannot rely on the filename address being valid or even returning the same data every time. This is the cause of a ton of behind-the-scenes and often redundant reindexing that slows nearly all other media management programs.</li>
<li>A filename is often—for <i>ridiculous</i> reasons—limited to a certain prohibitive character set; even when utf-8 is supported, some arbitrary ascii characters are usually not, and different localisations, operating systems and formatting conventions only make it worse.</p>
<li>Folders can offer context, but they are clunky and time-consuming to change. If you put each chapter of a comic in a different folder, for instance, reading several volumes in one sitting can be a pain. Nesting many folders adds navigation-latency and tends to induce less informative "04.jpg"-type filenames.</li>
</ul>
<p>So, the client tracks files by their <i>hash</i>.</p>
<p><i>BTW: when exporting files, the client names them by their hexadecimalised hash, like so: f099b5823f4e36a4bd6562812582f60e49e818cf445902b504b5533c6a5dad94.jpg. This will probably change to tag-munged in future.</i></p>
<p>Please do not tag your files with their exact original 'filename.jpg' on my public tag repo. <a href="http://www.youtube.com/watch?v=_yYS0ZZdsnA">Shed the concept of filenames as you would chains.</a></p>
<a name="hashes"><h3>hold up, what is a hash?</h3></a>
<p><a href="http://en.wikipedia.org/wiki/Hash_function">wiki</a></p>
<p>Hashes are a subject one usually has to be a software engineer to find interesting. If you don't care to digest the wiki page, the simple answer is that hashes are guaranteed unique names for things. It can be proven that f099b5823f4e36a4bd6562812582f60e49e818cf445902b504b5533c6a5dad94 refers to one particular file and no other. Hashes make excellent—if ugly—identifiers. In the client's normal operation, you will never encounter a file's hash; if you like a thumbnail, double-click it; the software handles the mathematics.</p>
<p><i>For those who </i>are<i> interested: hydrus uses SHA-256, which spits out 32-byte (256-bit) hashes. The software stores and searches over the hash densely, as 32 bytes, only encoding it to 64 hex characters when the user views it or copies to clipboard. SHA-256 is not perfect, but it is a great compromise candidate; it is secure for now, it is reasonably fast, it is available for most programming languages, and newer CPUs perform it more efficiently all the time. Maybe when NIST decides on the SHA-3 winner we will have a grand switch over.</i></p>
<a name="access_keys"><h3>hold up, what is an access key?</h3></a>
<p>The hydrus network's repositories do not use username/password, but instead a single combination identifier-password like this: <i>7ce4dbf18f7af8b420ee942bae42030aab344e91dc0e839260fcd71a4c9879e3</i></p>
<p>These hex numbers give you access to a particular account on a particular repository. They are long enough to be impossible to guess, and also randomly generated, so they reveal nothing personally identifying about you. Many people can use the same access key (and hence the same account) on a repository without consequence, although they will have to share bandwidth limits, and if one person screws around and gets the account banned, they will all lose access.</p>
<h3>why aren't my swfs showing?</h3>
<p>If an Internet Explorer "Navigation Cancelled" page appears whenever you click on a swf thumbnail, try installing Flash Player for Internet Explorer. Just having it installed for Firefox/Opera is not enough; you need the ActiveX component that comes with the specific IE version. Just boot IE and download/run the installer from Adobe's site.</p>
<a name="other_platforms"><h3>why shouldn't I use a more mature platform?</h3></a>
<p>Some applications like ACDSee try to make finding files easier than browsing explorer, but they are all-too-often:</p>
<p>
<ul>
<li>weighed down by noob-friendly 'features'</li>
<li>lacking any easy and open method of sharing files or tags</li>
<li>proprietary, untrustworthy, and expensive</li>
<li>good at one specific job, but only that</li>
<li>cluttered with inefficient database code, and perpetual reindexing</li>
<li>designed by people who think the internet is AOL.com</li>
</ul>
</p>
<p>Some websites like flickr and danbooru have crowd-sourced tags and offer fairly effective retrieval, but then <i>they</i> all-too-often have:</p>
<p>
<ul>
<li>choked bandwidth/server CPU</li>
<li>high search latency</li>
<li>degraded image quality</li>
<li>small results sets (e.g. 12 results to a page)</li>
<li>arbitrary obscenity rules</li>
<li>intrusive advertisement</li>
<li>unreliable access</li>
<li>uncertain privacy</li>
</ul>
</p>
<p>The hydrus network attempts to combine the privacy and low latency of local searching with the efficiency of crowd-sourcing.</p>
<a name="delays"><h3>why can my friend not see what I just uploaded?</h3></a>
<p>The repositories do not work like conventional search engines; it takes a short but predictable while for changes to propagate to other users.</p>
<p>Remember that the client's searches only ever happen over its local cache of what is on the repository. Those caches are updated about once a day, so any changes you make will be delayed for others until their next update occurs. At the moment, the update period is 100,000 seconds, which is about 1 day and 4 hours.</p>
</div>
</body>
</html>

261
help/getting_started_files.html
View File

@ -1,131 +1,132 @@
<html>
<head>
<title>getting started - files</title>
<link href="hydrus.ico" rel="shortcut icon" />
<link href="style.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div class="content">
<p><a href="introduction.html"><--- Back to the introduction</a></p>
<h3 class="warning">a warning</h3>
<p class="warning">This is the real internet, not babby AOL. The hydrus client gives you the power to screw up your life. If you want to do private sexy slideshows of your shy wife that's fine, but don't upload the pictures anywhere you don't absolutely trust and don't give them public tags that'll identify anyone. It is <b>impossible</b> to contain leaks of private information.</p>
<h3>the problem</h3>
<p>If you have ever seen something like this—</p>
<p><img src="pictures.png" title="After a while, I started just dropping everything in here unsorted. It would only grow, hungry and untouchable." /></p>
<p>—then you already know the problem: using a filesystem to manage a lot of images sucks.</p>
<p>Finding the right picture within a minute can be difficult. Finding all those by a particular artist or of a particular resolution within any reasonable time limit can be impossible. Adding new files into the whole mess is a further pain, and most operating systems bug out displaying folders with > 10,000 images.</p>
<h3>so, what does the hydrus client do?</h3>
<p>Let's first focus on storing and sharing files.</p>
<p>On booting the client for the first time, you will be faced with a blank screen and little idea of what to do next. I advise you simply drag-and-drop a folder with a hundred or so images onto the main window. After a little parsing, a dialog will appear affirming what you want to import. Ok that and a new page will open. Thumbnails will stream in as the software processes each file.</p>
<p><a href="import.png"><img src="import.png" width="683" height="384" /></a></p>
<p>The files are being imported into the client's database. <a href="faq.html#filenames">The client discards their filenames.</a></p>
<p>Notice your original folder and its files are untouched. You can move the originals somewhere else, delete them, and the client will still return searches fine. In the same way, you can delete from the client, and the original files will remain unchanged; import is a <b>copy</b>, not a move, operation. The client performs all its operations on its internal database. If you find yourself enjoying using the client and decide to completely switch over, you may delete the original files you import without worry. You can always export them back again later (albeit with different filenames).</p>
<p>Now:</p>
<ul class="bulletpoint">
<li>Click on a thumbnail; see what happens.</li>
<li>Now double- or middle-click it to go fullscreen. You can hit 'f' to switch between giving the fullcreen a frame or not. You can mouse-scroll or page up and down to scroll through the media, and double/middle-clicking again closes fullscreen. Hitting Enter/Return works just like double/middle click, as long as you have a thumbnail already focussed.</li>
<li>Try shift- or ctrl- selecting several files, and notice how the status bar at the bottom of the screen changes. Right click your selection.</li>
<li>Hit F9 to bring up a new page chooser. It won't show much right now, because you are just started. You can navigate it with the arrow keys, your numpad, or your mouse.</li>
<li>
<p>On the left of a normal 'file search' page is a text box with a large pop-up dropdown. It looks like this:</p>
<p><img src="autocomplete_dropdown_overlay.png" /></p>
<p>This is the autocomplete tag entry. It is where you put in the predicates to do a search. The dropdown window only appears when the text box is focussed. If the text box is empty, it will show a number of 'system' tags that let you search by non-tag metadata such as file size or animation duration. Typing in the text box will, when you have some tags in your database, show the tags that begin with whatever you type. The following (number) shows how many files have that tag, and hence how large the search result will be if you select that tag. You can scroll the dropdown list with the arrow keys or ctrl + scrollwheel, and then hit enter or just double click on a tag to enter it into the current search.</p>
<p>There are also several buttons to play with:</p>
<ul class="bulletpoint">
<li><b>include current/pending tags</b> - will determine whether the autocomplete results (and their counts) will be harvested from tags that currently exist, and/or the tags that are waiting to be sent to a service (more on this in <i>getting started with tags</i>).</li>
<li><b>searching immediately</b> - determines whether new searches will be performed as soon as you add or remove a search predicate, or whether the client will wait until you hit the button again. This is useful if you have a complicated query and don't want to wait for the search with every step.</li>
<li><b>file repository/tag repository</b> - determines the search domain for the autocomplete tags. Selecting 'all known files/tags' will union all known file or tag services together. Be careful selecting both 'all known files' and 'all known tags', as queries (and even just the dropdown!) will be delayed!</li>
</ul>
</li>
<li>Once you are happy with the dropdown, try hitting 'system:size', and maybe change the resultant dialog's &lt; to &gt; or the 100KB to 1MB.</li>
<li>You can remove from the list of 'active tags' above with a double-click, or by entering the same tag again through the dropdown.</li>
<li>Play with the system tags more if you like, and the sort-by dropdown. The collect-by dropdown (which collects by certain tags) will only do things if your files have appropriate tags.</li>
<li>To close a page's tab, middle click it.</li>
</ul>
<p>The client currently supports the following mimetypes:</p>
<ul>
<li><b>image/bmp</b> (.bmp - converted to image/png on import)</li>
<li><b>image/gif</b> (.gif)</li>
<li><b>image/png</b> (.png)</li>
<li><b>image/jpeg</b> (.jpg)</li>
<li><b>application/x-shockwave-flash</b> (.swf)</li>
<li><b>video/x-flv</b> (.flv)</li>
</ul>
<p>The client can also download files from several websites, including 4chan, many boorus, and gallery sites like deviant art. The different options are under F9->download.</p>
<p><a href="downloads.png"><img src="downloads.png" width="683" height="384" /></a></p>
<p>Most of them have similar interfaces. Paste the url or type the query your are interested in, and press enter.</p>
<p><a href="faq.html#filenames">FAQ: why not use filenames and folders?</a></p>
<h3>inbox and archiving</h3>
<p>the client sends newly imported/downloaded files to an <b>inbox</b> so you may more easily decide what to do with them. Inbox acts like a tag, matched by 'system:inbox'. A small envelope icon is drawn in the top corner of all inbox files.</p>
<p>If you are sure you want to keep a file long-term, you should <b>archive</b> it, which will remove it from the inbox. You can archive from your selected thumbnails' right-click menu, or by pressing F7.</p>
<p>Anything you do not want to keep should be deleted.</p>
<p>A quick way of doing this is—</p>
<h3>filtering</h3>
<p>Lets say you just downloaded a good thread, or perhaps you just imported an old folder of miscellany. You now have a whole bunch of files in your inbox—some good, some awful. You probably want to quickly go through them, saying <i>yes, yes, yes, no, yes, no, no, yes</i>, where <i>yes</i> means 'keep and archive' and <i>no</i> means 'delete this trash'. <b>Filtering</b> is the solution.</p>
<p>Select some thumbnails, and either choose filter from their right-click menu or hit F12. You will see this selection in fullscreen, with the following controls:</p>
<ul>
<li>Left-click, space, or F7: <b>keep and archive the file, move on</b></li>
<li>Right-click or delete: <b>delete the file, move on</b></li>
<li>Arrow key up: <b>Skip this file, move on</b></li>
<li>Middle-click or backspace: <b>I didn't mean that, go back one</b></li>
<li>Escape, return, or F12: <b>stop filtering now</b></li>
</ul>
<p>When done, you will be asked whether you want to commit your choices, forget them, or go back to filtering the current file.</p>
<p>Filtering saves time.</p>
<p>I have plans to make a filtering-like system to speed up certain kinds of tagging. Your thoughts would be appreciated.</p>
<h3>exporting and uploading</h3>
<p>There are many ways to export files from the client:</p>
<ul>
<li>
<p><b>raw export</b></p>
<p>Right clicking some files and selecting 'export all' will copy all those files from the database to your export folder, which is set in <i>file->options</i> and easily accessed in <i>file->open export folder</i>.</p>
<p>The files will have ugly filenames. It is on my to-do list to make them better, perhaps tag-based.</p>
<p>This is an ugly operation, but it is quick. It is best when you want to mass-export many thousands of files.</p>
</li>
<li>
<p><b>copy->copy all</b></p>
<p>Right clicking some files and selecting 'copy all' from the copy submenu will export the files to a temporary folder and copy their paths to your clipboard. You can then paste them wherever you like, just as with normal files. They will have the same ugly filenames as with a normal export.</p>
<p>This is a very quick operation, and can also be triggered by hitting Ctrl+C. It is best when you want to export just a few files somewhere for a temporary job.</p>
</li>
<li>
<p><b>copy->hash/hashes</b></p>
<p>Right clicking some files and selecting 'copy hash/hashes' will copy the files' unique identifiers to your clipboard.</p>
<p>You will not have to do this often. It is best when you want to tell someone else about a number of files without giving them the files.</p>
</li>
<li>
<p><b>copy->copy path/local url</b></p>
<p>Right clicking any file and selecting 'copy path' will copy the file's raw database path (install_path/db/client_files/[hash]) to your clipboard. 'copy local url' does the same, but with a localhost url in the form http://127.0.0.1:45865/file?hash=[hash].</p>
<p>These are most useful when you want to send a single file to another program. You can copy either of these addresses into a file open dialog like so:</p>
<p><a href="upload.png"><img src="upload.png" width="960" height="600" /></a></p>
<p>This works for file upload, opening a file in a graphics editor, or whatever. You can also paste into your browser's address bar, to check they are working.</p>
<p>The path method will always work, the url method will only work while the client is running.</p>
<p>The path method will send the file's hash as the filename, the url method will send something odd like file[7].</p>
<p>The path method will change your current working directory to your client's db directory (possibly annoying, if you have a lot of files), the url method will change your current working directory to temporary internet files.</p>
<p>I generally recommend you go with the url method.</p>
<p>If you use the path method to open a file inside an image editing program, try to remember to go 'save as' and give it a new filename! The client does not expect files inside its db directory to change.</p>
</li>
<li>
<p><b>dumping to imageboard</b></p>
<p>Right clicking some files and selecting 'dump all' will let you mass-dump them to an imageboard. You'll be asked which board you want to dump to and then taken to a new page:</p>
<p><a href="dumping.png"><img src="dumping.png" width="960" height="600" /></a></p>
<p>If you have a 4chan pass, you can authenticate the client in <i>services->manage 4chan pass</i>. Any new dump pages will no longer show the captcha window.</p>
<p>The client comes with all of 4chan's boards pre-configured. If you feel very brave and confident of your html-form-parsing skills, go <i>services->manage imageboards</i> and try to add some new sites.</p>
</li>
</ul>
<h3>sharing files</h3>
<p>The hydrus network has a service that lets clients share files anonymously, called a <i>file repository</i>.</p>
<p>It simply stores files in a big pool. Anyone who has an access key to the repository can see the pool's thumbnails and download anything they like. They may have permission to to upload to it as well. Admins can delete. I run a download-only file repository, which you are welcome to connect to to get a feel for the interface. Go <i>services->add, remove or edit services</i>.</p>
<p><img src="edit_repos_file_repo.png" /></p>
<ul><li>8f8a3685abc19e78a92ba61d84a0482b1cfac176fd853f46d93fe437a95e40a5@98.214.1.156:45872</li></ul>
<p>Then go <i>services->review services</i> to see your client synchronise with the repository's file list.</p>
<p><img src="review_services.png" /></p>
<p>Hit F9, and you'll see a new "files->" page. It works exactly like a local search, it just uses a different file list. Files you do not have will be drawn with a dark background, those you do will be drawn as normal:</p>
<p><a href="lib_rec.png"><img src="lib_rec.png" width="683" height="384" /></a></p>
<p>To download a file, double- or middle-click it, or select from the right click menu.</p>
<p>If you have permission to upload files to a particular repository, that option will appear in the right-click menu for any local files. Selecting this will pend them for batch uploading; just select from the new <i>pending</i> menu to effect the upload when you are ready.</p>
<h3>lastly</h3>
<p>The hydrus client is not an image-editing program, nor is it particularly intended for half-finished images. Think of it as a giant archive, a library, for everything excellent you have decided to store away.</p>
<p class="right"><a href="getting_started_tags.html">Now let's learn about tags! ----></a></p>
</div>
</body>
<html>
<head>
<title>getting started - files</title>
<link href="hydrus.ico" rel="shortcut icon" />
<link href="style.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div class="content">
<p><a href="introduction.html"><--- Back to the introduction</a></p>
<h3 class="warning">a warning</h3>
<p class="warning">This is the real internet, not babby AOL. The hydrus client gives you the power to screw up your life. If you want to do private sexy slideshows of your shy wife that's fine, but don't upload the pictures anywhere you don't absolutely trust and don't give them public tags that'll identify anyone. It is <b>impossible</b> to contain leaks of private information.</p>
<h3>the problem</h3>
<p>If you have ever seen something like this—</p>
<p><img src="pictures.png" title="After a while, I started just dropping everything in here unsorted. It would only grow, hungry and untouchable." /></p>
<p>—then you already know the problem: using a filesystem to manage a lot of images sucks.</p>
<p>Finding the right picture within a minute can be difficult. Finding all those by a particular artist or of a particular resolution within any reasonable time limit can be impossible. Adding new files into the whole mess is a further pain, and most operating systems bug out displaying folders with > 10,000 images.</p>
<h3>so, what does the hydrus client do?</h3>
<p>Let's first focus on storing and sharing files.</p>
<p>On booting the client for the first time, you will be faced with a blank screen and little idea of what to do next. I advise you simply drag-and-drop a folder with a hundred or so images onto the main window. After a little parsing, a dialog will appear affirming what you want to import. Ok that and a new page will open. Thumbnails will stream in as the software processes each file.</p>
<p><a href="import.png"><img src="import.png" width="683" height="384" /></a></p>
<p>The files are being imported into the client's database. <a href="faq.html#filenames">The client discards their filenames.</a></p>
<p>Notice your original folder and its files are untouched. You can move the originals somewhere else, delete them, and the client will still return searches fine. In the same way, you can delete from the client, and the original files will remain unchanged; import is a <b>copy</b>, not a move, operation. The client performs all its operations on its internal database. If you find yourself enjoying using the client and decide to completely switch over, you may delete the original files you import without worry. You can always export them back again later (albeit with different filenames).</p>
<p>Now:</p>
<ul class="bulletpoint">
<li>Click on a thumbnail; see what happens.</li>
<li>Now double- or middle-click it to go fullscreen. You can hit 'f' to switch between giving the fullcreen a frame or not. You can mouse-scroll or page up and down to scroll through the media, and double/middle-clicking again closes fullscreen. Hitting Enter/Return works just like double/middle click, as long as you have a thumbnail already focussed.</li>
<li>Try shift- or ctrl- selecting several files, and notice how the status bar at the bottom of the screen changes. Right click your selection.</li>
<li>Hit F9 to bring up a new page chooser. It won't show much right now, because you are just started. You can navigate it with the arrow keys, your numpad, or your mouse.</li>
<li>
<p>On the left of a normal 'file search' page is a text box with a large pop-up dropdown. It looks like this:</p>
<p><img src="autocomplete_dropdown_overlay.png" /></p>
<p>This is the autocomplete tag entry. It is where you put in the predicates to do a search. The dropdown window only appears when the text box is focussed. If the text box is empty, it will show a number of 'system' tags that let you search by non-tag metadata such as file size or animation duration. Typing in the text box will, when you have some tags in your database, show the tags that begin with whatever you type. The following (number) shows how many files have that tag, and hence how large the search result will be if you select that tag. You can scroll the dropdown list with the arrow keys or ctrl + scrollwheel, and then hit enter or just double click on a tag to enter it into the current search.</p>
<p>There are also several buttons to play with:</p>
<ul class="bulletpoint">
<li><b>include current/pending tags</b> - will determine whether the autocomplete results (and their counts) will be harvested from tags that currently exist, and/or the tags that are waiting to be sent to a service (more on this in <i>getting started with tags</i>).</li>
<li><b>searching immediately</b> - determines whether new searches will be performed as soon as you add or remove a search predicate, or whether the client will wait until you hit the button again. This is useful if you have a complicated query and don't want to wait for the search with every step.</li>
<li><b>file repository/tag repository</b> - determines the search domain for the autocomplete tags. Selecting 'all known files/tags' will union all known file or tag services together. Be careful selecting both 'all known files' and 'all known tags', as queries (and even just the dropdown!) will be delayed!</li>
</ul>
</li>
<li>Once you are happy with the dropdown, try hitting 'system:size', and maybe change the resultant dialog's &lt; to &gt; or the 100KB to 1MB.</li>
<li>You can remove from the list of 'active tags' above with a double-click, or by entering the same tag again through the dropdown.</li>
<li>Play with the system tags more if you like, and the sort-by dropdown. The collect-by dropdown (which collects by certain tags) will only do things if your files have appropriate tags.</li>
<li>To close a page's tab, middle click it.</li>
</ul>
<p>The client currently supports the following mimetypes:</p>
<ul>
<li><b>image/bmp</b> (.bmp - converted to image/png on import)</li>
<li><b>image/gif</b> (.gif)</li>
<li><b>image/png</b> (.png)</li>
<li><b>image/jpeg</b> (.jpg)</li>
<li><b>application/x-shockwave-flash</b> (.swf)</li>
<li><b>application/pdf</b> (.pdf)</li>
<li><b>video/x-flv</b> (.flv)</li>
</ul>
<p>The client can also download files from several websites, including 4chan, many boorus, and gallery sites like deviant art. The different options are under F9->download.</p>
<p><a href="downloads.png"><img src="downloads.png" width="683" height="384" /></a></p>
<p>Most of them have similar interfaces. Paste the url or type the query your are interested in, and press enter.</p>
<p><a href="faq.html#filenames">FAQ: why not use filenames and folders?</a></p>
<h3>inbox and archiving</h3>
<p>the client sends newly imported/downloaded files to an <b>inbox</b> so you may more easily decide what to do with them. Inbox acts like a tag, matched by 'system:inbox'. A small envelope icon is drawn in the top corner of all inbox files.</p>
<p>If you are sure you want to keep a file long-term, you should <b>archive</b> it, which will remove it from the inbox. You can archive from your selected thumbnails' right-click menu, or by pressing F7.</p>
<p>Anything you do not want to keep should be deleted.</p>
<p>A quick way of doing this is—</p>
<h3>filtering</h3>
<p>Lets say you just downloaded a good thread, or perhaps you just imported an old folder of miscellany. You now have a whole bunch of files in your inbox—some good, some awful. You probably want to quickly go through them, saying <i>yes, yes, yes, no, yes, no, no, yes</i>, where <i>yes</i> means 'keep and archive' and <i>no</i> means 'delete this trash'. <b>Filtering</b> is the solution.</p>
<p>Select some thumbnails, and either choose filter from their right-click menu or hit F12. You will see this selection in fullscreen, with the following controls:</p>
<ul>
<li>Left-click, space, or F7: <b>keep and archive the file, move on</b></li>
<li>Right-click or delete: <b>delete the file, move on</b></li>
<li>Arrow key up: <b>Skip this file, move on</b></li>
<li>Middle-click or backspace: <b>I didn't mean that, go back one</b></li>
<li>Escape, return, or F12: <b>stop filtering now</b></li>
</ul>
<p>When done, you will be asked whether you want to commit your choices, forget them, or go back to filtering the current file.</p>
<p>Filtering saves time.</p>
<p>I have plans to make a filtering-like system to speed up certain kinds of tagging. Your thoughts would be appreciated.</p>
<h3>exporting and uploading</h3>
<p>There are many ways to export files from the client:</p>
<ul>
<li>
<p><b>raw export</b></p>
<p>Right clicking some files and selecting 'export all' will copy all those files from the database to your export folder, which is set in <i>file->options</i> and easily accessed in <i>file->open export folder</i>.</p>
<p>The files will have ugly filenames. It is on my to-do list to make them better, perhaps tag-based.</p>
<p>This is an ugly operation, but it is quick. It is best when you want to mass-export many thousands of files.</p>
</li>
<li>
<p><b>copy->copy all</b></p>
<p>Right clicking some files and selecting 'copy all' from the copy submenu will export the files to a temporary folder and copy their paths to your clipboard. You can then paste them wherever you like, just as with normal files. They will have the same ugly filenames as with a normal export.</p>
<p>This is a very quick operation, and can also be triggered by hitting Ctrl+C. It is best when you want to export just a few files somewhere for a temporary job.</p>
</li>
<li>
<p><b>copy->hash/hashes</b></p>
<p>Right clicking some files and selecting 'copy hash/hashes' will copy the files' unique identifiers to your clipboard.</p>
<p>You will not have to do this often. It is best when you want to tell someone else about a number of files without giving them the files.</p>
</li>
<li>
<p><b>copy->copy path/local url</b></p>
<p>Right clicking any file and selecting 'copy path' will copy the file's raw database path (install_path/db/client_files/[hash]) to your clipboard. 'copy local url' does the same, but with a localhost url in the form http://127.0.0.1:45865/file?hash=[hash].</p>
<p>These are most useful when you want to send a single file to another program. You can copy either of these addresses into a file open dialog like so:</p>
<p><a href="upload.png"><img src="upload.png" width="960" height="600" /></a></p>
<p>This works for file upload, opening a file in a graphics editor, or whatever. You can also paste into your browser's address bar, to check they are working.</p>
<p>The path method will always work, the url method will only work while the client is running.</p>
<p>The path method will send the file's hash as the filename, the url method will send something odd like file[7].</p>
<p>The path method will change your current working directory to your client's db directory (possibly annoying, if you have a lot of files), the url method will change your current working directory to temporary internet files.</p>
<p>I generally recommend you go with the url method.</p>
<p>If you use the path method to open a file inside an image editing program, try to remember to go 'save as' and give it a new filename! The client does not expect files inside its db directory to change.</p>
</li>
<li>
<p><b>dumping to imageboard</b></p>
<p>Right clicking some files and selecting 'dump all' will let you mass-dump them to an imageboard. You'll be asked which board you want to dump to and then taken to a new page:</p>
<p><a href="dumping.png"><img src="dumping.png" width="960" height="600" /></a></p>
<p>If you have a 4chan pass, you can authenticate the client in <i>services->manage 4chan pass</i>. Any new dump pages will no longer show the captcha window.</p>
<p>The client comes with all of 4chan's boards pre-configured. If you feel very brave and confident of your html-form-parsing skills, go <i>services->manage imageboards</i> and try to add some new sites.</p>
</li>
</ul>
<h3>sharing files</h3>
<p>The hydrus network has a service that lets clients share files anonymously, called a <i>file repository</i>.</p>
<p>It simply stores files in a big pool. Anyone who has an access key to the repository can see the pool's thumbnails and download anything they like. They may have permission to to upload to it as well. Admins can delete. I run a download-only file repository, which you are welcome to connect to to get a feel for the interface. Go <i>services->add, remove or edit services</i>.</p>
<p><img src="edit_repos_file_repo.png" /></p>
<ul><li>8f8a3685abc19e78a92ba61d84a0482b1cfac176fd853f46d93fe437a95e40a5@98.214.1.156:45872</li></ul>
<p>Then go <i>services->review services</i> to see your client synchronise with the repository's file list.</p>
<p><img src="review_services.png" /></p>
<p>Hit F9, and you'll see a new "files->" page. It works exactly like a local search, it just uses a different file list. Files you do not have will be drawn with a dark background, those you do will be drawn as normal:</p>
<p><a href="lib_rec.png"><img src="lib_rec.png" width="683" height="384" /></a></p>
<p>To download a file, double- or middle-click it, or select from the right click menu.</p>
<p>If you have permission to upload files to a particular repository, that option will appear in the right-click menu for any local files. Selecting this will pend them for batch uploading; just select from the new <i>pending</i> menu to effect the upload when you are ready.</p>
<h3>lastly</h3>
<p>The hydrus client is not an image-editing program, nor is it particularly intended for half-finished images. Think of it as a giant archive, a library, for everything excellent you have decided to store away.</p>
<p class="right"><a href="getting_started_tags.html">Now let's learn about tags! ----></a></p>
</div>
</body>
</html>

93
help/updates.html
View File

@ -1,47 +1,48 @@
<html>
<head>
<title>updates</title>
<link href="hydrus.ico" rel="shortcut icon" />
<link href="style.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div class="content">
<h3>how the hydrus network synchronises</h3>
<p>The hydrus network does not work like regular client-server architectures.</p>
<p>The most important difference is its decentralisation of processing; rather than make an expensive http request every time it wants something, the client makes an all-inclusive synchronisation request about once a day and performs all searches on its local cache.</p>
<h3>so, how does the client make sure it has what it needs to do its searches?</h3>
<p>When the client contacts a repository, it downloads every single change that has occured since the last time it checked. It keeps all this data, and searches over whatever is appropriate to its own circumstances. If its local circumstances change (e.g. you import a thousand new files), it doesn't need to download anything more. A repository does not know anything about any particular client's circumstances.</p>
<h3>tell me more! use diagrams!</h3>
<p>These diagrams are a little old! 'librarium' is the old name for the client, and now there are multiple tag update caches, which are combined into a new table called 'active mappings'. I'll update them sooooometime.</p>
<p>tags:</p>
<p><a href="tag_sync_1.png"><img src="tag_sync_1.png" width="960" height="443" /></a></p>
<p><a href="tag_sync_2.png"><img src="tag_sync_2.png" width="960" height="443" /></a></p>
<p><a href="tag_sync_3.png"><img src="tag_sync_3.png" width="960" height="443" /></a></p>
<p><a href="tag_sync_4.png"><img src="tag_sync_4.png" width="960" height="443" /></a></p>
<p>files:</p>
<p><a href="file_sync_1.png"><img src="file_sync_1.png" width="960" height="443" /></a></p>
<p><a href="file_sync_2.png"><img src="file_sync_2.png" width="960" height="443" /></a></p>
<p><a href="file_sync_3.png"><img src="file_sync_3.png" width="960" height="443" /></a></p>
<p><a href="file_sync_4.png"><img src="file_sync_4.png" width="960" height="443" /></a></p>
<p><a href="file_sync_5.png"><img src="file_sync_5.png" width="960" height="443" /></a></p>
<p><a href="file_sync_6.png"><img src="file_sync_6.png" width="960" height="443" /></a></p>
<h3>the update request</h3>
<p>The main request looks like this:</p>
<ul>
<li>GET /update?begin=1298778949 HTTP/1.1</li>
</ul>
<p>Which is a standard http query. 'begin' is a timestamp telling the repository "please give me the update which starts with this timestamp" (begin=0 initialises). The repository answers in YAML, which you can review in include/HydrusConstants.py.</p>
<p>The update duration is currently 100,000 seconds.</p>
<h3>headers</h3>
<p>All requests (other than '/' and '/favicon'), should have something like the following:</p>
<ul>
<li>Authorization: hydrus_network 7ce4dbf18f7af8b420ee942bae42030aab344e91dc0e839260fcd71a4c9879e3</li>
<li>User-Agent: hydrus/6</p>
</ul>
<p>The user-agent doesn't have to be 'hydrus', but the network version afterwards has to match up, or you'll get an error.</p>
<h3>what about the other requests?</h3>
<p>I suggest you review the code for information on the other requests. HydrusServer.py does the parsing, and ProcessRequest in the databases does most of the actual magic. ConnectionToService in ClientConstants.py does the client-side request-bundling and response parsing. If you have detailed questions, you can always email me!</p>
<p>YAML is very important in the hydrus network. I love it. Just do some googling if you want to learn more, and play around with yaml.safe_dump and yaml.safe_load in the python console to get some hands-on experience.</p>
</div>
</body>
<html>
<head>
<title>updates</title>
<link href="hydrus.ico" rel="shortcut icon" />
<link href="style.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div class="content">
<h3>how the hydrus network synchronises</h3>
<p>The hydrus network does not work like regular client-server architectures.</p>
<p>The most important difference is its decentralisation of processing; rather than make an expensive http request every time it wants something, the client makes an all-inclusive synchronisation request about once a day and performs all searches on its local cache.</p>
<h3>so, how does the client make sure it has what it needs to do its searches?</h3>
<p>When the client contacts a repository, it downloads every single change that has occured since the last time it checked. It keeps all this data, and searches over whatever is appropriate to its own circumstances. If its local circumstances change (e.g. you import a thousand new files), it doesn't need to download anything more. A repository does not know anything about any particular client's circumstances.</p>
<h3>tell me more! use diagrams!</h3>
<p>These diagrams are a little old! 'librarium' is the old name for the client, and now there are multiple tag update caches, which are combined into a new table called 'active mappings'. I'll update them sooooometime.</p>
<p>tags:</p>
<p><a href="tag_sync_1.png"><img src="tag_sync_1.png" width="960" height="443" /></a></p>
<p><a href="tag_sync_2.png"><img src="tag_sync_2.png" width="960" height="443" /></a></p>
<p><a href="tag_sync_3.png"><img src="tag_sync_3.png" width="960" height="443" /></a></p>
<p><a href="tag_sync_4.png"><img src="tag_sync_4.png" width="960" height="443" /></a></p>
<p>files:</p>
<p><a href="file_sync_1.png"><img src="file_sync_1.png" width="960" height="443" /></a></p>
<p><a href="file_sync_2.png"><img src="file_sync_2.png" width="960" height="443" /></a></p>
<p><a href="file_sync_3.png"><img src="file_sync_3.png" width="960" height="443" /></a></p>
<p><a href="file_sync_4.png"><img src="file_sync_4.png" width="960" height="443" /></a></p>
<p><a href="file_sync_5.png"><img src="file_sync_5.png" width="960" height="443" /></a></p>
<p><a href="file_sync_6.png"><img src="file_sync_6.png" width="960" height="443" /></a></p>
<h3>the update request</h3>
<p>The main request looks like this:</p>
<ul>
<li>GET /update?begin=1298778949 HTTP/1.1</li>
</ul>
<p>Which is a standard http query. 'begin' is a timestamp telling the repository "please give me the update which starts with this timestamp" (begin=0 initialises). The repository answers in YAML, which you can review in include/HydrusConstants.py.</p>
<p>The update duration is currently 100,000 seconds.</p>
<h3>headers</h3>
<p>The repository's requests need a user agent and a session key, which is just a cookie. You can fetch a new session key like so:</p>
<ul>
<li>GET /session_key HTTP/1.1</li>
<li>Authorization: hydrus_network 7ce4dbf18f7af8b420ee942bae42030aab344e91dc0e839260fcd71a4c9879e3</li>
<li>User-Agent: hydrus/NETWORK_VERSION</p>
</ul>
<p>Where NETWORK_VERSION is the current version, such as '9', and the authorisation is your access key. The user-agent doesn't have to be 'hydrus', but the network version afterwards has to match up with the server's, or you'll get a 426 error.</p>
<h3>what about the other requests?</h3>
<p>I suggest you review the code for information on the other requests. HydrusServer.py does the parsing, and ProcessRequest in the databases does most of the actual magic. ConnectionToService in ClientConstants.py does the client-side request-bundling and response parsing. If you have detailed questions, you can always email me!</p>
<p>YAML is very important in the hydrus network. I love it. Just do some googling if you want to learn more, and play around with yaml.safe_dump and yaml.safe_load in the python console to get some hands-on experience.</p>
</div>
</body>
</html>

419
include/ClientConstants.py
View File

@ -703,6 +703,19 @@ class AutocompleteMatchesCounted():
def GetMatches( self, search ): return [ ( match, self._matches_to_count[ match ] ) for match in self._matches if HC.SearchEntryMatchesTag( search, match ) ]
class AutocompleteMatchesPredicates():
def __init__( self, predicates ):
self._predicates = predicates
def cmp_func( x, y ): return cmp( x.GetCount(), y.GetCount() )
self._predicates.sort( cmp = cmp_func, reverse = True )
def GetMatches( self, search ): return [ predicate for predicate in self._predicates if HC.SearchEntryMatchesPredicate( search, predicate ) ]
class Booru( HC.HydrusYAMLBase ):
yaml_tag = u'!Booru'
@ -1621,7 +1634,7 @@ class FileQueryResult():
class FileSearchContext():
def __init__( self, file_service_identifier = LOCAL_FILE_SERVICE_IDENTIFIER, tag_service_identifier = NULL_SERVICE_IDENTIFIER, include_current_tags = True, include_pending_tags = True, raw_predicates = [] ):
def __init__( self, file_service_identifier = LOCAL_FILE_SERVICE_IDENTIFIER, tag_service_identifier = NULL_SERVICE_IDENTIFIER, include_current_tags = True, include_pending_tags = True, predicates = [] ):
self._file_service_identifier = file_service_identifier
self._tag_service_identifier = tag_service_identifier
@ -1629,20 +1642,43 @@ class FileSearchContext():
self._include_current_tags = include_current_tags
self._include_pending_tags = include_pending_tags
self._raw_predicates = raw_predicates
self._predicates = predicates
raw_system_predicates = [ predicate for predicate in raw_predicates if predicate.startswith( 'system:' ) ]
system_predicates = [ predicate for predicate in predicates if predicate.GetPredicateType() == HC.PREDICATE_TYPE_SYSTEM ]
self._system_predicates = FileSystemPredicates( raw_system_predicates )
self._system_predicates = FileSystemPredicates( system_predicates )
raw_tags = [ predicate for predicate in raw_predicates if not predicate.startswith( 'system:' ) ]
tag_predicates = [ predicate for predicate in predicates if predicate.GetPredicateType() == HC.PREDICATE_TYPE_TAG ]
self._tags_to_include = [ tag for tag in raw_tags if not tag.startswith( '-' ) ]
self._tags_to_exclude = [ tag[1:] for tag in raw_tags if tag.startswith( '-' ) ]
self._tags_to_include = []
self._tags_to_exclude = []
for predicate in tag_predicates:
( operator, tag ) = predicate.GetValue()
if operator == '+': self._tags_to_include.append( tag )
elif operator == '-': self._tags_to_exclude.append( tag )
namespace_predicates = [ predicate for predicate in predicates if predicate.GetPredicateType() == HC.PREDICATE_TYPE_NAMESPACE ]
self._namespaces_to_include = []
self._namespaces_to_exclude = []
for predicate in namespace_predicates:
( operator, namespace ) = predicate.GetValue()
if operator == '+': self._namespaces_to_include.append( namespace )
elif operator == '-': self._namespaces_to_exclude.append( namespace )
def GetFileServiceIdentifier( self ): return self._file_service_identifier
def GetRawPredicates( self ): return self._raw_predicates
def GetNamespacesToExclude( self ): return self._namespaces_to_exclude
def GetNamespacesToInclude( self ): return self._namespaces_to_include
def GetPredicates( self ): return self._predicates
def GetSystemPredicates( self ): return self._system_predicates
def GetTagServiceIdentifier( self ): return self._tag_service_identifier
def GetTagsToExclude( self ): return self._tags_to_exclude
@ -1682,13 +1718,10 @@ class FileSystemPredicates():
self._predicates[ self.RATIO ] = []
self._predicates[ self.REPOSITORIES ] = []
self._inbox = 'system:inbox' in system_predicates
self._archive = 'system:archive' in system_predicates
self._local = 'system:local' in system_predicates
self._not_local = 'system:not local' in system_predicates
self._inbox = False
self._archive = False
self._local = False
self._not_local = False
self._num_tags_zero = False
self._num_tags_nonzero = False
@ -1706,6 +1739,9 @@ class FileSystemPredicates():
self._min_height = None
self._height = None
self._max_height = None
self._min_num_words = None
self._num_words = None
self._max_num_words = None
self._min_duration = None
self._duration = None
self._max_duration = None
@ -1726,286 +1762,171 @@ class FileSystemPredicates():
for predicate in system_predicates:
if predicate.startswith( 'system:hash=' ):
( system_predicate_type, info ) = predicate.GetValue()
if system_predicate_type == HC.SYSTEM_PREDICATE_TYPE_INBOX: self._inbox = True
if system_predicate_type == HC.SYSTEM_PREDICATE_TYPE_ARCHIVE: self._archive = True
if system_predicate_type == HC.SYSTEM_PREDICATE_TYPE_LOCAL: self._local = True
if system_predicate_type == HC.SYSTEM_PREDICATE_TYPE_NOT_LOCAL: self._not_local = True
if system_predicate_type == HC.SYSTEM_PREDICATE_TYPE_HASH:
try:
hash = predicate[12:].decode( 'hex' )
self._hash = hash
except: raise Exception( 'I could not parse the hash predicate.' )
hash = info
self._hash = hash
if predicate.startswith( 'system:age' ):
if system_predicate_type == HC.SYSTEM_PREDICATE_TYPE_AGE:
try:
( operator, years, months, days ) = info
timestamp = int( time.time() ) - ( ( ( ( ( years * 12 ) + months ) * 30 ) + days ) * 86400 )
# this is backwards because we are talking about age, not timestamp
if operator == '<': self._min_timestamp = timestamp
elif operator == '>': self._max_timestamp = timestamp
elif operator == u'\u2248':
condition = predicate[10]
self._min_timestamp = int( timestamp * 0.85 )
self._max_timestamp = int( timestamp * 1.15 )
if condition not in ( '<', '>', u'\u2248' ): raise Exception()
age = predicate[11:]
years = 0
months = 0
days = 0
if 'y' in age:
( years, age ) = age.split( 'y' )
years = int( years )
if 'm' in age:
( months, age ) = age.split( 'm' )
months = int( months )
if 'd' in age:
( days, age ) = age.split( 'd' )
days = int( days )
timestamp = int( time.time() ) - ( ( ( ( ( years * 12 ) + months ) * 30 ) + days ) * 86400 )
# this is backwards because we are talking about age, not timestamp
if condition == '<': self._min_timestamp = timestamp
elif condition == '>': self._max_timestamp = timestamp
elif condition == u'\u2248':
self._min_timestamp = int( timestamp * 0.85 )
self._max_timestamp = int( timestamp * 1.15 )
except: raise Exception( 'I could not parse the age predicate.' )
if predicate.startswith( 'system:mime' ):
if system_predicate_type == HC.SYSTEM_PREDICATE_TYPE_MIME:
try:
mime = predicate[12:]
if mime == 'image': self._mimes = HC.IMAGES
elif mime == 'application': self._mimes = HC.APPLICATIONS
else: self._mimes = ( HC.mime_enum_lookup[ mime ], )
except: raise Exception( 'I could not parse the mime predicate.' )
mimes = info
if type( mimes ) == int: mimes = ( mimes, )
self._mimes = mimes
if predicate.startswith( 'system:duration' ):
if system_predicate_type == HC.SYSTEM_PREDICATE_TYPE_DURATION:
try:
( operator, duration ) = info
if operator == '<': self._max_duration = duration
elif operator == '>': self._min_duration = duration
elif operator == '=': self._duration = duration
elif operator == u'\u2248':
condition = predicate[15]
self._min_duration = int( duration * 0.85 )
self._max_duration = int( duration * 1.15 )
if condition not in ( '>', '<', '=', u'\u2248' ): raise Exception()
duration = int( predicate[16:] )
if duration >= 0:
if condition == '<': self._max_duration = duration
elif condition == '>': self._min_duration = duration
elif condition == '=': self._duration = duration
elif condition == u'\u2248':
self._min_duration = int( duration * 0.85 )
self._max_duration = int( duration * 1.15 )
except: raise Exception( 'I could not parse the duration predicate.' )
if predicate.startswith( 'system:rating' ):
if system_predicate_type == HC.SYSTEM_PREDICATE_TYPE_RATING:
try:
# system:rating:[service_name][operator][value]
stuff_i_care_about = predicate[14:]
operators = [ '<', u'\u2248', '=', '>' ]
for operator in operators:
if operator in stuff_i_care_about:
( service_name, value ) = stuff_i_care_about.split( operator )
self._ratings_predicates.append( ( service_name, operator, value ) )
break
except: raise Exception( 'I could not parse the ratio predicate.' )
( service_identifier, operator, value ) = info
self._ratings_predicates.append( ( service_identifier, operator, value ) )
if predicate.startswith( 'system:ratio' ):
if system_predicate_type == HC.SYSTEM_PREDICATE_TYPE_RATIO:
try:
condition = predicate[12]
if condition not in ( '=', u'\u2248' ): raise Exception()
ratio = predicate[13:]
( width, height ) = ratio.split( ':', 1 )
width = float( width )
height = float( height )
if width > 0 and height > 0:
if condition == '=': self._predicates[ self.RATIO ].append( ( equals, width / height ) )
elif condition == u'\u2248': self._predicates[ self.RATIO ].append( ( about_equals, width / height ) )
except: raise Exception( 'I could not parse the ratio predicate.' )
( operator, ratio ) = info
if operator == '=': self._predicates[ self.RATIO ].append( ( equals, ratio ) )
elif operator == u'\u2248': self._predicates[ self.RATIO ].append( ( about_equals, ratio ) )
if predicate.startswith( 'system:size' ):
if system_predicate_type == HC.SYSTEM_PREDICATE_TYPE_SIZE:
try:
( operator, size, unit ) = info
size = size * unit
if operator == '<': self._max_size = size
elif operator == '>': self._min_size = size
elif operator == '=': self._size = size
elif operator == u'\u2248':
condition = predicate[11]
self._min_size = int( size * 0.85 )
self._max_size = int( size * 1.15 )
if condition not in ( '>', '<', '=', u'\u2248' ): raise Exception()
size = int( predicate[12:-2] )
multiplier = predicate[-2]
if multiplier == 'k': multiplier = 1000
elif multiplier == 'K': multiplier = 1024
elif multiplier == 'm': multiplier = 1000000
elif multiplier == 'M': multiplier = 1048576
elif multiplier == 'g': multiplier = 1000000000
elif multiplier == 'G': multiplier = 1073741824
else:
multiplier = 1
size = int( predicate[12:-1] )
size = size * multiplier
bB = predicate[-1]
if bB not in ( 'b', 'B' ): raise Exception()
if bB == 'b': size = size / 8
if condition == '<': self._max_size = size
elif condition == '>': self._min_size = size
elif condition == '=': self._size = size
elif condition == u'\u2248':
self._min_size = int( size * 0.85 )
self._max_size = int( size * 1.15 )
except: raise Exception( 'I could not parse the size predicate.' )
if predicate.startswith( 'system:numtags' ):
if system_predicate_type == HC.SYSTEM_PREDICATE_TYPE_NUM_TAGS:
try:
( operator, num_tags ) = info
if operator == '<': self._predicates[ self.NUM_TAGS ].append( ( lessthan, num_tags ) )
elif operator == '>':
condition = predicate[14]
self._predicates[ self.NUM_TAGS ].append( ( greaterthan, num_tags ) )
if condition not in ( '>', '<', '=' ): raise Exception()
if num_tags == 0: self._num_tags_nonzero = True
num_tags = int( predicate[15:] )
elif operator == '=':
if num_tags >= 0:
if condition == '<': self._predicates[ self.NUM_TAGS ].append( ( lessthan, num_tags ) )
elif condition == '>':
self._predicates[ self.NUM_TAGS ].append( ( greaterthan, num_tags ) )
if num_tags == 0: self._num_tags_nonzero = True
elif condition == '=':
self._predicates[ self.NUM_TAGS ].append( ( equals, num_tags ) )
if num_tags == 0: self._num_tags_zero = True
self._predicates[ self.NUM_TAGS ].append( ( equals, num_tags ) )
if num_tags == 0: self._num_tags_zero = True
except: raise Exception( 'I could not parse the numtags predicate.' )
if predicate.startswith( 'system:width' ):
if system_predicate_type == HC.SYSTEM_PREDICATE_TYPE_WIDTH:
try:
( operator, width ) = info
if operator == '<': self._max_width = width
elif operator == '>': self._min_width = width
elif operator == '=': self._width = width
elif operator == u'\u2248':
condition = predicate[12]
self._min_width = int( width * 0.85 )
self._max_width = int( width * 1.15 )
if condition not in ( '>', '<', '=', u'\u2248' ): raise Exception()
width = int( predicate[13:] )
if width >= 0:
if condition == '<': self._max_width = width
elif condition == '>': self._min_width = width
elif condition == '=': self._width = width
elif condition == u'\u2248':
self._min_width = int( width * 0.85 )
self._max_width = int( width * 1.15 )
except: raise Exception( 'I could not parse the width predicate.' )
if predicate.startswith( 'system:height' ):
if system_predicate_type == HC.SYSTEM_PREDICATE_TYPE_HEIGHT:
try:
( operator, height ) = info
if operator == '<': self._max_height = height
elif operator == '>': self._min_height = height
elif operator == '=': self._height = height
elif operator == u'\u2248':
condition = predicate[13]
self._min_height = int( height * 0.85 )
self._max_height = int( height * 1.15 )
if condition not in ( '>', '<', '=', u'\u2248' ): raise Exception()
height = int( predicate[14:] )
if height >= 0:
if condition == '<': self._max_height = height
elif condition == '>': self._min_height = height
elif condition == '=': self._height = height
elif condition == u'\u2248':
self._min_height = int( height * 0.85 )
self._max_height = int( height * 1.15 )
except: raise Exception( 'I could not parse the height predicate.' )
if predicate.startswith( 'system:limit=' ):
if system_predicate_type == HC.SYSTEM_PREDICATE_TYPE_NUM_WORDS:
try: self._limit = int( predicate[13:] )
except: raise Exception( 'I could not parse the limit predicate.' )
( operator, num_words ) = info
if operator == '<': self._max_num_words = num_words
elif operator == '>': self._min_num_words = num_words
elif operator == '=': self._num_words = num_words
elif operator == u'\u2248':
self._min_num_words = int( num_words * 0.85 )
self._max_num_words = int( num_words * 1.15 )
if predicate.startswith( 'system:not_uploaded_to:' ): self._file_repositories_to_exclude.append( predicate[23:] )
if predicate.startswith( 'system:similar_to=' ):
if system_predicate_type == HC.SYSTEM_PREDICATE_TYPE_LIMIT:
try:
( hash, max_hamming ) = predicate[18:].split( u'\u2248', 1 )
self._similar_to = ( hash.decode( 'hex' ), int( max_hamming ) )
except: raise Exception( 'I could not parse the similar to predicate.' )
limit = info
self._limit = limit
if system_predicate_type == HC.SYSTEM_PREDICATE_TYPE_NOT_UPLOADED_TO:
service_identifier = info
self._file_repositories_to_exclude.append( service_identifier )
if system_predicate_type == HC.SYSTEM_PREDICATE_TYPE_SIMILAR_TO:
( hash, max_hamming ) = info
self._similar_to = ( hash, max_hamming )
@ -2028,9 +1949,9 @@ class FileSystemPredicates():
return True
def GetFileRepositoryNamesToExclude( self ): return self._file_repositories_to_exclude
def GetFileRepositoriesToExclude( self ): return self._file_repositories_to_exclude
def GetInfo( self ): return ( self._hash, self._min_size, self._size, self._max_size, self._mimes, self._min_timestamp, self._max_timestamp, self._min_width, self._width, self._max_width, self._min_height, self._height, self._max_height, self._min_duration, self._duration, self._max_duration )
def GetInfo( self ): return ( self._hash, self._min_size, self._size, self._max_size, self._mimes, self._min_timestamp, self._max_timestamp, self._min_width, self._width, self._max_width, self._min_height, self._height, self._max_height, self._min_num_words, self._num_words, self._max_num_words, self._min_duration, self._duration, self._max_duration )
def GetLimit( self ): return self._limit
@ -2052,6 +1973,8 @@ class FileSystemPredicates():
def OkFirstRound( self, width, height ):
if len( self._predicates[ self.RATIO ] ) > 0 and ( width is None or height is None ): return False
if False in ( function( float( width ) / float( height ), arg ) for ( function, arg ) in self._predicates[ self.RATIO ] ): return False
return True

250
include/ClientDB.py
View File

@ -1716,8 +1716,6 @@ class ServiceDB( FileDB, MessageDB, TagDB, RatingDB ):
except: pass
local_files_hashes = { hash.decode( 'hex' ) for hash in dircache.listdir( HC.CLIENT_FILES_DIR ) }
for hash in local_files_hashes & deletee_hashes: os.remove( HC.CLIENT_FILES_DIR + os.path.sep + hash.encode( 'hex' ) )
# perceptual_hashes and thumbs
@ -1803,12 +1801,15 @@ class ServiceDB( FileDB, MessageDB, TagDB, RatingDB ):
tags_to_include = search_context.GetTagsToInclude()
tags_to_exclude = search_context.GetTagsToExclude()
namespaces_to_include = search_context.GetNamespacesToInclude()
namespaces_to_exclude = search_context.GetNamespacesToExclude()
include_current_tags = search_context.IncludeCurrentTags()
include_pending_tags = search_context.IncludePendingTags()
sql_predicates = [ 'service_id = ' + str( file_service_id ) ]
( hash, min_size, size, max_size, mimes, min_timestamp, max_timestamp, min_width, width, max_width, min_height, height, max_height, min_duration, duration, max_duration ) = system_predicates.GetInfo()
( hash, min_size, size, max_size, mimes, min_timestamp, max_timestamp, min_width, width, max_width, min_height, height, max_height, min_num_words, num_words, max_num_words, min_duration, duration, max_duration ) = system_predicates.GetInfo()
if min_size is not None: sql_predicates.append( 'size > ' + str( min_size ) )
if size is not None: sql_predicates.append( 'size = ' + str( size ) )
@ -1836,6 +1837,10 @@ class ServiceDB( FileDB, MessageDB, TagDB, RatingDB ):
if height is not None: sql_predicates.append( 'height = ' + str( height ) )
if max_height is not None: sql_predicates.append( 'height < ' + str( max_height ) )
if min_num_words is not None: sql_predicates.append( 'num_words > ' + str( min_num_words ) )
if num_words is not None: sql_predicates.append( 'num_words = ' + str( num_words ) )
if max_num_words is not None: sql_predicates.append( 'num_words < ' + str( max_num_words ) )
if min_duration is not None: sql_predicates.append( 'duration > ' + str( min_duration ) )
if duration is not None:
@ -1844,9 +1849,19 @@ class ServiceDB( FileDB, MessageDB, TagDB, RatingDB ):
if max_duration is not None: sql_predicates.append( 'duration < ' + str( max_duration ) )
if len( tags_to_include ) > 0:
if len( tags_to_include ) > 0 or len( namespaces_to_include ) > 0:
query_hash_ids = HC.IntelligentMassIntersect( ( self._GetHashIdsFromTag( c, file_service_identifier, tag_service_identifier, tag, include_current_tags, include_pending_tags ) for tag in tags_to_include ) )
query_hash_ids = None
if len( tags_to_include ) > 0: query_hash_ids = HC.IntelligentMassIntersect( ( self._GetHashIdsFromTag( c, file_service_identifier, tag_service_identifier, tag, include_current_tags, include_pending_tags ) for tag in tags_to_include ) )
if len( namespaces_to_include ) > 0:
namespace_query_hash_ids = HC.IntelligentMassIntersect( ( self._GetHashIdsFromNamespace( c, file_service_identifier, tag_service_identifier, namespace, include_current_tags, include_pending_tags ) for namespace in namespaces_to_include ) )
if query_hash_ids is None: query_hash_ids = namespace_query_hash_ids
else: query_hash_ids.intersection_update( namespace_query_hash_ids )
if len( sql_predicates ) > 1: query_hash_ids.intersection_update( [ id for ( id, ) in c.execute( 'SELECT hash_id FROM files_info WHERE ' + ' AND '.join( sql_predicates ) + ';' ) ] )
@ -1914,20 +1929,22 @@ class ServiceDB( FileDB, MessageDB, TagDB, RatingDB ):
exclude_query_hash_ids = HC.IntelligentMassUnion( [ self._GetHashIdsFromTag( c, file_service_identifier, tag_service_identifier, tag, include_current_tags, include_pending_tags ) for tag in tags_to_exclude ] )
exclude_query_hash_ids.update( HC.IntelligentMassUnion( [ self._GetHashIdsFromNamespace( c, file_service_identifier, tag_service_identifier, namespace, include_current_tags, include_pending_tags ) for namespace in namespaces_to_exclude ] ) )
if file_service_type == HC.FILE_REPOSITORY and self._options[ 'exclude_deleted_files' ]: exclude_query_hash_ids.update( [ hash_id for ( hash_id, ) in c.execute( 'SELECT hash_id FROM deleted_files WHERE service_id = ?;', ( self._local_file_service_id, ) ) ] )
query_hash_ids.difference_update( exclude_query_hash_ids )
for name_to_exclude in system_predicates.GetFileRepositoryNamesToExclude():
for service_identifier in system_predicates.GetFileRepositoriesToExclude():
( service_id, ) = c.execute( 'SELECT service_id FROM services WHERE type = ? AND name = ?;', ( HC.FILE_REPOSITORY, name_to_exclude ) ).fetchone()
service_id = self._GetServiceId( c, service_identifier )
query_hash_ids.difference_update( [ hash_id for ( hash_id, ) in c.execute( 'SELECT hash_id FROM files_info WHERE service_id = ?;', ( service_id, ) ) ] )
for ( service_name, operator, value ) in system_predicates.GetRatingsPredicates():
for ( service_identifier, operator, value ) in system_predicates.GetRatingsPredicates():
service_id = self._GetServiceId( c, service_name )
service_id = self._GetServiceId( c, service_identifier )
if value == 'rated': query_hash_ids.intersection_update( [ hash_id for ( hash_id, ) in c.execute( 'SELECT hash_id FROM local_ratings WHERE service_id = ?;', ( service_id, ) ) ] )
elif value == 'not rated': query_hash_ids.difference_update( [ hash_id for ( hash_id, ) in c.execute( 'SELECT hash_id FROM local_ratings WHERE service_id = ?;', ( service_id, ) ) ] )
@ -2113,13 +2130,71 @@ class ServiceDB( FileDB, MessageDB, TagDB, RatingDB ):
[ tags_to_count.update( { ( 1, tag_id ) : num_tags } ) for ( namespace_id, tag_id, num_tags ) in results if namespace_id != 1 and tag_id in unnamespaced_tag_ids ]
matches = CC.AutocompleteMatchesCounted( { self._GetNamespaceTag( c, namespace_id, tag_id ) : num_tags for ( ( namespace_id, tag_id ), num_tags ) in tags_to_count.items() if num_tags > 0 } )
matches = CC.AutocompleteMatchesPredicates( [ HC.Predicate( HC.PREDICATE_TYPE_TAG, ( '+', self._GetNamespaceTag( c, namespace_id, tag_id ) ), num_tags ) for ( ( namespace_id, tag_id ), num_tags ) in tags_to_count.items() if num_tags > 0 ] )
return matches
def _GetFavouriteCustomFilterActions( self, c ): return dict( c.execute( 'SELECT name, actions FROM favourite_custom_filter_actions;' ).fetchall() )
def _GetHashIdsFromNamespace( self, c, file_service_identifier, tag_service_identifier, namespace, include_current_tags, include_pending_tags ):
hash_ids = set()
if file_service_identifier == CC.NULL_SERVICE_IDENTIFIER:
if tag_service_identifier == CC.NULL_SERVICE_IDENTIFIER:
current_tables_phrase = 'active_mappings'
pending_tables_phrase = 'active_pending_mappings'
current_predicates_phrase = ''
pending_predicates_phrase = ''
else:
tag_service_id = self._GetServiceId( c, tag_service_identifier )
current_tables_phrase = 'mappings'
pending_tables_phrase = 'pending_mappings'
current_predicates_phrase = 'service_id = ' + str( tag_service_id ) + ' AND '
pending_predicates_phrase = 'service_id = ' + str( tag_service_id ) + ' AND '
else:
file_service_id = self._GetServiceId( c, file_service_identifier )
if tag_service_identifier == CC.NULL_SERVICE_IDENTIFIER:
current_tables_phrase = '( active_mappings, files_info USING ( hash_id ) )'
pending_tables_phrase = '( active_pending_mappings, files_info USING ( hash_id ) )'
current_predicates_phrase = 'service_id = ' + str( file_service_id ) + ' AND '
pending_predicates_phrase = 'service_id = ' + str( file_service_id ) + ' AND '
else:
tag_service_id = self._GetServiceId( c, tag_service_identifier )
# we have to do a crazy join because of the nested joins, which wipe out table-namespaced identifiers like mappings.service_id, replacing them with useless stuff like service_id:1
current_tables_phrase = '( mappings, files_info ON ( mappings.hash_id = files_info.hash_id AND mappings.service_id = ' + str( tag_service_id ) + ' AND files_info.service_id = ' + str( file_service_id ) + ' ) )'
pending_tables_phrase = '( pending_mappings, files_info ON ( pending_mappings.hash_id = files_info.hash_id AND pending_mappings.service_id = ' + str( tag_service_id ) + ' AND files_info.service_id = ' + str( file_service_id ) + ' ) )'
current_predicates_phrase = ''
pending_predicates_phrase = ''
if include_current_tags: hash_ids.update( [ id for ( id, ) in c.execute( 'SELECT hash_id FROM namespaces, ' + current_tables_phrase + ' USING ( namespace_id ) WHERE ' + current_predicates_phrase + 'namespace = ?;', ( namespace, ) ) ] )
if include_pending_tags: hash_ids.update( [ id for ( id, ) in c.execute( 'SELECT hash_id FROM namespaces, ' + pending_tables_phrase + ' USING ( namespace_id ) WHERE ' + pending_predicates_phrase + 'namespace = ?;', ( namespace, ) ) ] )
return hash_ids
def _GetHashIdsFromTag( self, c, file_service_identifier, tag_service_identifier, tag, include_current_tags, include_pending_tags ):
hash_ids = set()
@ -2471,7 +2546,7 @@ class ServiceDB( FileDB, MessageDB, TagDB, RatingDB ):
media_results.append( CC.MediaResult( ( hash, inbox, size, mime, timestamp, width, height, duration, num_frames, num_words, tags_cdpp, file_service_identifiers_cdpp, local_ratings, remote_ratings ) ) )
return CC.FileQueryResult( file_service_identifier, search_context.GetRawPredicates(), media_results )
return CC.FileQueryResult( file_service_identifier, search_context.GetPredicates(), media_results )
def _GetMediaResultsFromHashes( self, c, search_context, hashes ):
@ -2893,99 +2968,59 @@ class ServiceDB( FileDB, MessageDB, TagDB, RatingDB ):
predicates = []
if service_type == HC.NULL_SERVICE:
if service_type in ( HC.NULL_SERVICE, HC.TAG_REPOSITORY, HC.LOCAL_TAG ):
service_info = self._GetServiceInfoSpecific( c, service_id, service_type, { HC.SERVICE_INFO_NUM_FILES } )
num_everything = service_info[ HC.SERVICE_INFO_NUM_FILES ]
predicates.append( ( u'system:everything', num_everything ) )
predicates.append( HC.Predicate( HC.PREDICATE_TYPE_SYSTEM, ( HC.SYSTEM_PREDICATE_TYPE_EVERYTHING, None ), num_everything ) )
predicates.extend( [ ( predicate, None ) for predicate in [ u'system:untagged', u'system:numtags', u'system:hash' ] ] )
predicates.extend( [ HC.Predicate( HC.PREDICATE_TYPE_SYSTEM, ( system_predicate_type, None ), None ) for system_predicate_type in [ HC.SYSTEM_PREDICATE_TYPE_UNTAGGED, HC.SYSTEM_PREDICATE_TYPE_NUM_TAGS, HC.SYSTEM_PREDICATE_TYPE_HASH ] ] )
# num local, would be great
# num inbox, would be great
# num local would be great
# num inbox would be great
# we can't guarantee knowing files_info, so only have untagged and numtags
elif service_type == HC.TAG_REPOSITORY:
service_info = self._GetServiceInfoSpecific( c, service_id, service_type, { HC.SERVICE_INFO_NUM_FILES } )
num_everything = service_info[ HC.SERVICE_INFO_NUM_FILES ]
predicates.append( ( u'system:everything', num_everything ) )
predicates.extend( [ ( predicate, None ) for predicate in [ u'system:untagged', u'system:numtags', u'system:hash' ] ] )
# num local, would be great
# num inbox, would be great
elif service_type == HC.LOCAL_TAG:
service_info = self._GetServiceInfoSpecific( c, service_id, service_type, { HC.SERVICE_INFO_NUM_FILES } )
num_everything = service_info[ HC.SERVICE_INFO_NUM_FILES ]
predicates.append( ( u'system:everything', num_everything ) )
predicates.extend( [ ( predicate, None ) for predicate in [ u'system:untagged', u'system:numtags', u'system:hash' ] ] )
# num local, would be great
# num inbox, would be great
elif service_type == HC.LOCAL_FILE:
elif service_type in ( HC.LOCAL_FILE, HC.FILE_REPOSITORY ):
service_info = self._GetServiceInfoSpecific( c, service_id, service_type, { HC.SERVICE_INFO_NUM_FILES, HC.SERVICE_INFO_NUM_INBOX } )
num_everything = service_info[ HC.SERVICE_INFO_NUM_FILES ]
num_inbox = service_info[ HC.SERVICE_INFO_NUM_INBOX ]
if service_type == HC.FILE_REPOSITORY:
if self._options[ 'exclude_deleted_files' ]:
( num_everything_deleted, ) = c.execute( 'SELECT COUNT( * ) FROM files_info, deleted_files USING ( hash_id ) WHERE files_info.service_id = ? AND deleted_files.service_id = ?;', ( service_id, self._local_file_service_id ) ).fetchone()
num_everything -= num_everything_deleted
num_inbox = service_info[ HC.SERVICE_INFO_NUM_INBOX ]
num_archive = num_everything - num_inbox
predicates.append( ( u'system:everything', num_everything ) )
predicates.append( HC.Predicate( HC.PREDICATE_TYPE_SYSTEM, ( HC.SYSTEM_PREDICATE_TYPE_EVERYTHING, None ), num_everything ) )
if num_inbox > 0:
predicates.append( ( u'system:inbox', num_inbox ) )
predicates.append( ( u'system:archive', num_archive ) )
predicates.append( HC.Predicate( HC.PREDICATE_TYPE_SYSTEM, ( HC.SYSTEM_PREDICATE_TYPE_INBOX, None ), num_inbox ) )
predicates.append( HC.Predicate( HC.PREDICATE_TYPE_SYSTEM, ( HC.SYSTEM_PREDICATE_TYPE_ARCHIVE, None ), num_archive ) )
predicates.extend( [ ( predicate, None ) for predicate in [ u'system:untagged', u'system:numtags', u'system:limit', u'system:size', u'system:age', u'system:hash', u'system:width', u'system:height', u'system:ratio', u'system:duration', u'system:mime', u'system:rating', u'system:similar_to' ] ] )
for service_identifier in self._GetServiceIdentifiers( c, ( HC.FILE_REPOSITORY, ) ): predicates.append( ( u'system:not_uploaded_to:' + service_identifier.GetName(), None ) )
elif service_type == HC.FILE_REPOSITORY:
service_info = self._GetServiceInfoSpecific( c, service_id, service_type, { HC.SERVICE_INFO_NUM_FILES, HC.SERVICE_INFO_NUM_INBOX } )
num_everything = service_info[ HC.SERVICE_INFO_NUM_FILES ]
num_inbox = service_info[ HC.SERVICE_INFO_NUM_INBOX ]
if self._options[ 'exclude_deleted_files' ]:
if service_type == HC.FILE_REPOSITORY:
( num_everything_deleted, ) = c.execute( 'SELECT COUNT( * ) FROM files_info, deleted_files USING ( hash_id ) WHERE files_info.service_id = ? AND deleted_files.service_id = ?;', ( service_id, self._local_file_service_id ) ).fetchone()
( num_local, ) = c.execute( 'SELECT COUNT( * ) FROM files_info AS remote_files_info, files_info USING ( hash_id ) WHERE remote_files_info.service_id = ? AND files_info.service_id = ?;', ( service_id, self._local_file_service_id ) ).fetchone()
num_everything -= num_everything_deleted
num_not_local = num_everything - num_local
predicates.append( HC.Predicate( HC.PREDICATE_TYPE_SYSTEM, ( HC.SYSTEM_PREDICATE_TYPE_LOCAL, None ), num_local ) )
predicates.append( HC.Predicate( HC.PREDICATE_TYPE_SYSTEM, ( HC.SYSTEM_PREDICATE_TYPE_NOT_LOCAL, None ), num_not_local ) )
( num_local, ) = c.execute( 'SELECT COUNT( * ) FROM files_info AS remote_files_info, files_info USING ( hash_id ) WHERE remote_files_info.service_id = ? AND files_info.service_id = ?;', ( service_id, self._local_file_service_id ) ).fetchone()
predicates.extend( [ HC.Predicate( HC.PREDICATE_TYPE_SYSTEM, ( system_predicate_type, None ), None ) for system_predicate_type in [ HC.SYSTEM_PREDICATE_TYPE_UNTAGGED, HC.SYSTEM_PREDICATE_TYPE_NUM_TAGS, HC.SYSTEM_PREDICATE_TYPE_LIMIT, HC.SYSTEM_PREDICATE_TYPE_SIZE, HC.SYSTEM_PREDICATE_TYPE_AGE, HC.SYSTEM_PREDICATE_TYPE_HASH, HC.SYSTEM_PREDICATE_TYPE_WIDTH, HC.SYSTEM_PREDICATE_TYPE_HEIGHT, HC.SYSTEM_PREDICATE_TYPE_RATIO, HC.SYSTEM_PREDICATE_TYPE_DURATION, HC.SYSTEM_PREDICATE_TYPE_NUM_WORDS, HC.SYSTEM_PREDICATE_TYPE_MIME, HC.SYSTEM_PREDICATE_TYPE_RATING, HC.SYSTEM_PREDICATE_TYPE_SIMILAR_TO ] ] )
num_not_local = num_everything - num_local
num_archive = num_local - num_inbox
predicates.append( ( u'system:everything', num_everything ) )
if num_inbox > 0:
predicates.append( ( u'system:inbox', num_inbox ) )
predicates.append( ( u'system:archive', num_archive ) )
predicates.append( ( u'system:local', num_local ) )
predicates.append( ( u'system:not local', num_not_local ) )
predicates.extend( [ ( predicate, None ) for predicate in [ u'system:untagged', u'system:numtags', u'system:limit', u'system:size', u'system:age', u'system:hash', u'system:width', u'system:height', u'system:ratio', u'system:duration', u'system:mime', u'system:rating', u'system:similar_to' ] ] )
predicates.extend( [ HC.Predicate( HC.PREDICATE_TYPE_SYSTEM, ( HC.SYSTEM_PREDICATE_TYPE_NOT_UPLOADED_TO, service_identifier ), None ) for service_identifier in self._GetServiceIdentifiers( c, ( HC.FILE_REPOSITORY, ) ) ] )
return predicates
@ -4358,6 +4393,7 @@ class DB( ServiceDB ):
HC.DAEMONWorker( 'DownloadFiles', self.DAEMONDownloadFiles, ( 'notify_new_downloads', 'notify_new_permissions' ) )
HC.DAEMONWorker( 'DownloadThumbnails', self.DAEMONDownloadThumbnails, ( 'notify_new_permissions', 'notify_new_thumbnails' ) )
HC.DAEMONWorker( 'ResizeThumbnails', self.DAEMONResizeThumbnails, () )
HC.DAEMONWorker( 'SynchroniseAccounts', self.DAEMONSynchroniseAccounts, ( 'notify_new_services', 'permissions_are_stale' ) )
HC.DAEMONWorker( 'SynchroniseMessages', self.DAEMONSynchroniseMessages, ( 'notify_new_permissions', 'notify_check_messages' ), period = 60 )
HC.DAEMONWorker( 'SynchroniseRepositories', self.DAEMONSynchroniseRepositories, ( 'notify_new_permissions', ) )
@ -4653,8 +4689,9 @@ class DB( ServiceDB ):
system_predicates[ 'local_rating_numerical' ] = ( 0, 3 )
system_predicates[ 'local_rating_like' ] = 0
system_predicates[ 'ratio' ] = ( 0, 16, 9 )
system_predicates[ 'size' ] = ( 0, 200, 3 )
system_predicates[ 'size' ] = ( 0, 200, 1 )
system_predicates[ 'width' ] = ( 1, 1920 )
system_predicates[ 'num_words' ] = ( 0, 30000 )
CLIENT_DEFAULT_OPTIONS[ 'file_system_predicates' ] = system_predicates
@ -4895,6 +4932,21 @@ class DB( ServiceDB ):
c.execute( 'CREATE TABLE hydrus_sessions ( service_id INTEGER PRIMARY KEY REFERENCES services ON DELETE CASCADE, session_key BLOB_BYTES, expiry INTEGER );' )
if version < 63:
( self._options, ) = c.execute( 'SELECT options FROM options;' ).fetchone()
system_predicates = self._options[ 'file_system_predicates' ]
( sign, size, unit ) = system_predicates[ 'size' ]
system_predicates[ 'size' ] = ( sign, size, 1 )
system_predicates[ 'num_words' ] = ( 0, 30000 )
c.execute( 'UPDATE options SET options = ?;', ( self._options, ) )
unknown_account = CC.GetUnknownAccount()
unknown_account.MakeStale()
@ -6085,6 +6137,44 @@ class DB( ServiceDB ):
def DAEMONFlushServiceUpdates( self, update_log ): self.Write( 'service_updates', HC.HIGH_PRIORITY, update_log )
def DAEMONResizeThumbnails( self ):
all_thumbnail_paths = dircache.listdir( HC.CLIENT_THUMBNAILS_DIR )
full_size_thumbnail_paths = { path for path in all_thumbnail_paths if not path.endswith( '_resized' ) }
resized_thumbnail_paths = { path for path in all_thumbnail_paths if path.endswith( '_resized' ) }
thumbnail_paths_to_render = full_size_thumbnail_paths.difference( resized_thumbnail_paths )
i = 0
limit = max( 100, len( thumbnail_paths_to_render ) / 10 )
for thumbnail_path in thumbnail_paths_to_render:
try:
with open( HC.CLIENT_THUMBNAILS_DIR + os.path.sep + thumbnail_path, 'rb' ) as f: thumbnail = f.read()
thumbnail_resized = HydrusImageHandling.GenerateThumbnailFileFromFile( thumbnail, self._options[ 'thumbnail_dimensions' ] )
thumbnail_resized_path_to = thumbnail_path + '_resized'
with open( HC.CLIENT_THUMBNAILS_DIR + os.path.sep + thumbnail_resized_path_to, 'wb' ) as f: f.write( thumbnail_resized )
except: print( traceback.format_exc() )
time.sleep( 1 )
i += 1
if i > limit: break
if HC.shutdown: break
def DAEMONSynchroniseAccounts( self ):
services = self.Read( 'services', HC.LOW_PRIORITY, HC.RESTRICTED_SERVICES )

8
include/ClientGUI.py
View File

@ -801,13 +801,13 @@ class FrameGUI( ClientGUICommon.Frame ):
def _NewPageQuery( self, service_identifier, tags = [] ):
def _NewPageQuery( self, service_identifier, initial_media_results = [], initial_predicates = [] ):
if service_identifier is None: service_identifier = ClientGUIDialogs.SelectServiceIdentifier( service_types = ( HC.FILE_REPOSITORY, ) )
if service_identifier is not None:
new_page = ClientGUIPages.PageQuery( self._notebook, service_identifier, tags )
new_page = ClientGUIPages.PageQuery( self._notebook, service_identifier, initial_media_results = initial_media_results, initial_predicates = initial_predicates )
self._notebook.AddPage( new_page, 'files', select = True )
@ -1120,7 +1120,7 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
def NewPagePetitions( self, service_identifier ): self._NewPagePetitions( service_identifier )
def NewPageQuery( self, service_identifier, tags = [] ): self._NewPageQuery( service_identifier, tags = tags )
def NewPageQuery( self, service_identifier, initial_media_results = [], initial_predicates = [] ): self._NewPageQuery( service_identifier, initial_media_results = initial_media_results, initial_predicates = initial_predicates )
def NewPageThreadDumper( self, hashes ):
@ -1139,7 +1139,7 @@ The password is cleartext here but obscured in the entry dialog. Enter a blank p
def NewSimilarTo( self, file_service_identifier, hash ): self._NewPageQuery( file_service_identifier, [ 'system:similar_to=' + hash.encode( 'hex' ) + u'\u2248' + '5' ] )
def NewSimilarTo( self, file_service_identifier, hash ): self._NewPageQuery( file_service_identifier, [ HC.Predicate( HC.PREDICATE_TYPE_SYSTEM, ( HC.SYSTEM_PREDICATE_TYPE_SIMILAR_TO, ( hash, 5 ) ), None ) ] )
def RefreshAcceleratorTable( self ):

14
include/ClientGUICanvas.py
View File

@ -1108,6 +1108,12 @@ class CanvasFullscreenMediaListBrowser( CanvasFullscreenMediaList ):
def EventShowMenu( self, event ):
services = wx.GetApp().Read( 'services' )
local_ratings_services = [ service for service in services if service.GetServiceIdentifier().GetType() in ( HC.LOCAL_RATING_LIKE, HC.LOCAL_RATING_NUMERICAL ) ]
i_can_post_ratings = len( local_ratings_services ) > 0
self._last_drag_coordinates = None # to stop successive right-click drag warp bug
menu = wx.Menu()
@ -1141,6 +1147,12 @@ class CanvasFullscreenMediaListBrowser( CanvasFullscreenMediaList ):
menu.AppendSeparator()
menu.Append( CC.MENU_EVENT_ID_TO_ACTION_CACHE.GetId( 'manage_tags' ), 'manage tags' )
if i_can_post_ratings: menu.Append( CC.MENU_EVENT_ID_TO_ACTION_CACHE.GetId( 'manage_ratings' ), 'manage ratings' )
menu.AppendSeparator()
if self._current_media.HasInbox(): menu.Append( CC.MENU_EVENT_ID_TO_ACTION_CACHE.GetId( 'archive' ), '&archive' )
if self._current_media.HasArchive(): menu.Append( CC.MENU_EVENT_ID_TO_ACTION_CACHE.GetId( 'inbox' ), 'return to &inbox' )
menu.Append( CC.MENU_EVENT_ID_TO_ACTION_CACHE.GetId( 'delete', CC.LOCAL_FILE_SERVICE_IDENTIFIER ), '&delete' )
@ -1842,7 +1854,7 @@ class RatingsFilterFrame( ClientGUICommon.Frame ):
rating = local_ratings.GetRating( self._service_identifier )
if other_min == 0.0: against_string += ' - dislike'
if rating == 0.0: against_string += ' - dislike'
else: against_string += ' - like'

164
include/ClientGUICommon.py
View File

@ -256,7 +256,14 @@ class AutoCompleteDropdown( wx.TextCtrl ):
def EventText( self, event ):
self._lag_timer.Start( 100, wx.TIMER_ONE_SHOT )
num_chars = len( self.GetValue() )
if num_chars == 0: lag = 0
elif num_chars == 1: lag = 400
elif num_chars == 2: lag = 200
else: lag = 100
self._lag_timer.Start( lag, wx.TIMER_ONE_SHOT )
class AutoCompleteDropdownContacts( AutoCompleteDropdown ):
@ -405,7 +412,7 @@ class AutoCompleteDropdownTags( AutoCompleteDropdown ):
matches = self._GenerateMatches()
self._dropdown_list.SetTags( matches )
self._dropdown_list.SetPredicates( matches )
self._current_matches = matches
@ -551,8 +558,18 @@ class AutoCompleteDropdownTagsRead( AutoCompleteDropdownTags ):
raw_entry = self.GetValue()
if raw_entry.startswith( '-' ): search_text = raw_entry[1:]
else: search_text = raw_entry
if raw_entry.startswith( '-' ):
operator = '-'
search_text = raw_entry[1:]
else:
operator = '+'
search_text = raw_entry
search_text = HC.CleanTag( search_text )
@ -590,9 +607,9 @@ class AutoCompleteDropdownTagsRead( AutoCompleteDropdownTags ):
if len( half_complete_tag ) >= num_first_letters:
if must_do_a_search or half_complete_tag[ : num_first_letters ] != self._first_letters:
if must_do_a_search or self._first_letters == '' or not half_complete_tag.startswith( self._first_letters ):
self._first_letters = half_complete_tag[ : num_first_letters ]
self._first_letters = half_complete_tag
media = self._media_callable()
@ -626,16 +643,21 @@ class AutoCompleteDropdownTagsRead( AutoCompleteDropdownTags ):
tags_to_count = collections.Counter( absolutely_all_tags_flat )
self._cached_results = CC.AutocompleteMatchesCounted( tags_to_count )
self._cached_results = CC.AutocompleteMatchesPredicates( [ HC.Predicate( HC.PREDICATE_TYPE_TAG, ( operator, tag ), count ) for ( tag, count ) in tags_to_count.items() ] )
matches = self._cached_results.GetMatches( half_complete_tag )
if raw_entry.startswith( '-' ): matches = [ ( '-' + tag, count ) for ( tag, count ) in matches ]
else: matches = []
if self._current_namespace != '': matches.insert( 0, HC.Predicate( HC.PREDICATE_TYPE_NAMESPACE, ( operator, namespace ), None ) )
for match in matches:
if match.GetPredicateType() == HC.PREDICATE_TYPE_TAG: match.SetOperator( operator )
return matches
@ -688,7 +710,18 @@ class AutoCompleteDropdownTagsWrite( AutoCompleteDropdownTags ):
self._dropdown_window.SetSizer( vbox )
def _BroadcastChoice( self, predicate ): self._chosen_tag_callable( predicate )
def _BroadcastChoice( self, predicate ):
if predicate is None: broadcast = None
else:
( operator, tag ) = predicate.GetValue()
broadcast = tag
self._chosen_tag_callable( broadcast )
def _GenerateMatches( self ):
@ -732,9 +765,9 @@ class AutoCompleteDropdownTagsWrite( AutoCompleteDropdownTags ):
if len( half_complete_tag ) >= num_first_letters:
if must_do_a_search or half_complete_tag[ : num_first_letters ] != self._first_letters:
if must_do_a_search or self._first_letters == '' or not half_complete_tag.startswith( self._first_letters ):
self._first_letters = half_complete_tag[ : num_first_letters ]
self._first_letters = half_complete_tag
self._cached_results = wx.GetApp().Read( 'autocomplete_tags', file_service_identifier = self._file_service_identifier, tag_service_identifier = self._tag_service_identifier, half_complete_tag = search_text )
@ -743,19 +776,17 @@ class AutoCompleteDropdownTagsWrite( AutoCompleteDropdownTags ):
else: matches = []
predicate = HC.Predicate( HC.PREDICATE_TYPE_TAG, ( '+', search_text ), 0 )
try:
tags_in_order = [ tag for ( tag, count ) in matches ]
predicate = matches[ matches.index( predicate ) ]
index = tags_in_order.index( search_text )
matches.remove( predicate )
match = matches[ index ]
matches.remove( match )
matches.insert( 0, match )
except: matches.insert( 0, ( search_text, 0 ) )
except: pass
matches.insert( 0, predicate )
return matches
@ -2496,32 +2527,31 @@ class TagsBoxActiveOnly( TagsBox ):
self._callable = callable
self._matches = {}
self._predicates = {}
def _Activate( self, tag ): self._callable( tag )
def _Activate( self, predicate ): self._callable( predicate )
def SetTags( self, matches ):
def SetPredicates( self, predicates ):
if matches != self._matches:
if predicates != self._predicates:
self._matches = matches
self._predicates = predicates
self._ordered_strings = []
self._strings_to_terms = {}
for ( tag, count ) in matches:
for predicate in predicates:
if count is None: tag_string = tag
else: tag_string = tag + ' (' + HC.ConvertIntToPrettyString( count ) + ')'
tag_string = predicate.GetUnicode()
self._ordered_strings.append( tag_string )
self._strings_to_terms[ tag_string ] = tag
self._strings_to_terms[ tag_string ] = predicate
self._TextsHaveChanged()
if len( matches ) > 0: self._Select( 0 )
if len( predicates ) > 0: self._Select( 0 )
@ -2546,7 +2576,12 @@ class TagsBoxCPP( TagsBox ):
HC.pubsub.sub( self, 'ChangeTagRepository', 'change_tag_repository' )
def _Activate( self, tag ): HC.pubsub.pub( 'add_predicate', self._page_key, tag )
def _Activate( self, tag ):
predicate = HC.Predicate( HC.PREDICATE_TYPE_TAG, ( '+', tag ), None )
HC.pubsub.pub( 'add_predicate', self._page_key, predicate )
def _SortTags( self ):
@ -2886,50 +2921,51 @@ class TagsBoxPredicates( TagsBox ):
def _Activate( self, tag ): HC.pubsub.pub( 'remove_predicate', self._page_key, tag )
def ActivatePredicate( self, tag ):
if tag in self._ordered_strings:
self._ordered_strings.remove( tag )
del self._strings_to_terms[ tag ]
else:
if tag == 'system:inbox' and 'system:archive' in self._ordered_strings: self._ordered_strings.remove( 'system:archive' )
elif tag == 'system:archive' and 'system:inbox' in self._ordered_strings: self._ordered_strings.remove( 'system:inbox' )
elif tag == 'system:local' and 'system:not local' in self._ordered_strings: self._ordered_strings.remove( 'system:not local' )
elif tag == 'system:not local' and 'system:local' in self._ordered_strings: self._ordered_strings.remove( 'system:local' )
self._ordered_strings.append( tag )
self._strings_to_terms[ tag ] = tag
self._ordered_strings.sort()
self._TextsHaveChanged()
def _Activate( self, predicate ): HC.pubsub.pub( 'remove_predicate', self._page_key, predicate )
def AddPredicate( self, predicate ):
self._ordered_strings.append( predicate )
self._strings_to_terms[ predicate ] = predicate
predicate = predicate.GetCountlessCopy()
predicate_string = predicate.GetUnicode()
inbox_predicate = HC.Predicate( HC.PREDICATE_TYPE_SYSTEM, ( HC.SYSTEM_PREDICATE_TYPE_INBOX, None ), None )
archive_predicate = HC.Predicate( HC.PREDICATE_TYPE_SYSTEM, ( HC.SYSTEM_PREDICATE_TYPE_ARCHIVE, None ), None )
if predicate == inbox_predicate and self.HasPredicate( archive_predicate ): self.RemovePredicate( archive_predicate )
elif predicate == archive_predicate and self.HasPredicate( inbox_predicate ): self.RemovePredicate( inbox_predicate )
local_predicate = HC.Predicate( HC.PREDICATE_TYPE_SYSTEM, ( HC.SYSTEM_PREDICATE_TYPE_LOCAL, None ), None )
not_local_predicate = HC.Predicate( HC.PREDICATE_TYPE_SYSTEM, ( HC.SYSTEM_PREDICATE_TYPE_NOT_LOCAL, None ), None )
if predicate == local_predicate and self.HasPredicate( not_local_predicate ): self.RemovePredicate( not_local_predicate )
elif predicate == not_local_predicate and self.HasPredicate( local_predicate ): self.RemovePredicate( local_predicate )
self._ordered_strings.append( predicate_string )
self._strings_to_terms[ predicate_string ] = predicate
self._ordered_strings.sort()
self._TextsHaveChanged()
def GetPredicates( self ): return self._ordered_strings
def GetPredicates( self ): return self._strings_to_terms.values()
def HasPredicate( self, predicate ): return predicate in self._ordered_strings
def HasPredicate( self, predicate ): return predicate in self._strings_to_terms.values()
def RemovePredicate( self, predicate ):
self._ordered_strings.remove( predicate )
del self._strings_to_terms[ predicate ]
self._TextsHaveChanged()
for ( s, existing_predicate ) in self._strings_to_terms.items():
if existing_predicate == predicate:
self._ordered_strings.remove( s )
del self._strings_to_terms[ s ]
self._TextsHaveChanged()
break

298
include/ClientGUIDialogs.py
View File

@ -944,9 +944,12 @@ class DialogInputFileSystemPredicate( Dialog ):
self._sign = wx.Choice( self, choices=[ '<', u'\u2248', '>' ] )
self._sign.SetSelection( sign )
self._years = wx.SpinCtrl( self, initial = years, max = 30 )
self._months = wx.SpinCtrl( self, initial = months, max = 60 )
self._days = wx.SpinCtrl( self, initial = days, max = 90 )
self._years = wx.SpinCtrl( self, max = 30 )
self._years.SetValue( years )
self._months = wx.SpinCtrl( self, max = 60 )
self._months.SetValue( months )
self._days = wx.SpinCtrl( self, max = 90 )
self._days.SetValue( days )
self._ok = wx.Button( self, label='Ok' )
self._ok.Bind( wx.EVT_BUTTON, self.EventOk )
@ -1186,7 +1189,7 @@ class DialogInputFileSystemPredicate( Dialog ):
( sign, num_tags ) = system_predicates[ 'num_tags' ]
self._sign = wx.Choice( self, choices=[ '<', '=', '>' ] )
self._sign = wx.Choice( self, choices=[ '<', u'\u2248', '=', '>' ] )
self._sign.SetSelection( sign )
self._num_tags = wx.SpinCtrl( self, max = 2000 )
@ -1201,7 +1204,7 @@ class DialogInputFileSystemPredicate( Dialog ):
hbox = wx.BoxSizer( wx.HORIZONTAL )
hbox.AddF( wx.StaticText( self, label='system:numtags' ), FLAGS_MIXED )
hbox.AddF( wx.StaticText( self, label='system:num_tags' ), FLAGS_MIXED )
hbox.AddF( self._sign, FLAGS_MIXED )
hbox.AddF( self._num_tags, FLAGS_MIXED )
hbox.AddF( self._ok, FLAGS_MIXED )
@ -1220,6 +1223,46 @@ class DialogInputFileSystemPredicate( Dialog ):
InitialisePanel()
def NumWords():
def InitialiseControls():
( sign, num_words ) = system_predicates[ 'num_words' ]
self._sign = wx.Choice( self, choices=[ '<', u'\u2248', '=', '>' ] )
self._sign.SetSelection( sign )
self._num_words = wx.SpinCtrl( self, max = 1000000 )
self._num_words.SetValue( num_words )
self._ok = wx.Button( self, label='Ok' )
self._ok.Bind( wx.EVT_BUTTON, self.EventOk )
self._ok.SetForegroundColour( ( 0, 128, 0 ) )
def InitialisePanel():
hbox = wx.BoxSizer( wx.HORIZONTAL )
hbox.AddF( wx.StaticText( self, label='system:num_words' ), FLAGS_MIXED )
hbox.AddF( self._sign, FLAGS_MIXED )
hbox.AddF( self._num_words, FLAGS_MIXED )
hbox.AddF( self._ok, FLAGS_MIXED )
self.SetSizer( hbox )
( x, y ) = self.GetEffectiveMinSize()
self.SetInitialSize( ( x, y ) )
Dialog.__init__( self, parent, 'enter number of words predicate' )
InitialiseControls()
InitialisePanel()
def Rating():
def InitialiseControls():
@ -1357,7 +1400,7 @@ class DialogInputFileSystemPredicate( Dialog ):
self._size = wx.SpinCtrl( self, max = 1048576 )
self._size.SetValue( size )
self._unit = wx.Choice( self, choices=[ 'b', 'B', 'Kb', 'KB', 'Mb', 'MB', 'Gb', 'GB' ] )
self._unit = wx.Choice( self, choices=[ 'B', 'KB', 'MB', 'GB' ] )
self._unit.SetSelection( unit )
self._ok = wx.Button( self, label='Ok' )
@ -1460,7 +1503,7 @@ class DialogInputFileSystemPredicate( Dialog ):
self.SetInitialSize( ( x, y ) )
Dialog.__init__( self, parent, 'enter duration predicate' )
Dialog.__init__( self, parent, 'enter similar to predicate' )
InitialiseControls()
@ -1473,18 +1516,19 @@ class DialogInputFileSystemPredicate( Dialog ):
self._type = type
if self._type == 'system:age': Age()
elif self._type == 'system:duration': Duration()
elif self._type == 'system:hash': Hash()
elif self._type == 'system:height': Height()
elif self._type == 'system:limit': Limit()
elif self._type == 'system:mime': Mime()
elif self._type == 'system:numtags': NumTags()
elif self._type == 'system:rating': Rating()
elif self._type == 'system:ratio': Ratio()
elif self._type == 'system:size': Size()
elif self._type == 'system:width': Width()
elif self._type == 'system:similar_to': SimilarTo()
if self._type == HC.SYSTEM_PREDICATE_TYPE_AGE: Age()
elif self._type == HC.SYSTEM_PREDICATE_TYPE_DURATION: Duration()
elif self._type == HC.SYSTEM_PREDICATE_TYPE_HASH: Hash()
elif self._type == HC.SYSTEM_PREDICATE_TYPE_HEIGHT: Height()
elif self._type == HC.SYSTEM_PREDICATE_TYPE_LIMIT: Limit()
elif self._type == HC.SYSTEM_PREDICATE_TYPE_MIME: Mime()
elif self._type == HC.SYSTEM_PREDICATE_TYPE_NUM_TAGS: NumTags()
elif self._type == HC.SYSTEM_PREDICATE_TYPE_RATING: Rating()
elif self._type == HC.SYSTEM_PREDICATE_TYPE_RATIO: Ratio()
elif self._type == HC.SYSTEM_PREDICATE_TYPE_SIZE: Size()
elif self._type == HC.SYSTEM_PREDICATE_TYPE_WIDTH: Width()
elif self._type == HC.SYSTEM_PREDICATE_TYPE_SIMILAR_TO: SimilarTo()
elif self._type == HC.SYSTEM_PREDICATE_TYPE_NUM_WORDS: NumWords()
self._hidden_cancel_button = wx.Button( self, id = wx.ID_CANCEL, label = 'cancel', size = ( 0, 0 ) )
self._hidden_cancel_button.Bind( wx.EVT_BUTTON, self.EventCancel )
@ -1523,13 +1567,85 @@ class DialogInputFileSystemPredicate( Dialog ):
def EventOk( self, event ):
if self._type == 'system:rating':
if self._type == HC.SYSTEM_PREDICATE_TYPE_AGE: info = ( self._sign.GetStringSelection(), self._years.GetValue(), self._months.GetValue(), self._days.GetValue() )
elif self._type == HC.SYSTEM_PREDICATE_TYPE_DURATION: info = ( self._sign.GetStringSelection(), self._duration_s.GetValue() * 1000 + self._duration_ms.GetValue() )
elif self._type == HC.SYSTEM_PREDICATE_TYPE_HASH:
hex_filter = lambda c: c in '0123456789abcdef'
hash = filter( hex_filter, self._hash.GetValue() )
if len( hash ) == 0: hash == '00'
elif len( hash ) % 2 == 1: hash += '0' # since we are later decoding to byte
info = hash.decode( 'hex' )
elif self._type == HC.SYSTEM_PREDICATE_TYPE_HEIGHT: info = ( self._sign.GetStringSelection(), self._height.GetValue() )
elif self._type == HC.SYSTEM_PREDICATE_TYPE_LIMIT: info = self._limit.GetValue()
elif self._type == HC.SYSTEM_PREDICATE_TYPE_MIME: info = self._mime_type.GetClientData( self._mime_type.GetSelection() )
elif self._type == HC.SYSTEM_PREDICATE_TYPE_NUM_TAGS: info = ( self._sign.GetStringSelection(), self._num_tags.GetValue() )
elif self._type == HC.SYSTEM_PREDICATE_TYPE_NUM_WORDS: info = ( self._sign.GetStringSelection(), self._num_words.GetValue() )
elif self._type == HC.SYSTEM_PREDICATE_TYPE_RATING:
id = event.GetId()
if id == HC.LOCAL_RATING_LIKE: self._type = 'system:rating_like'
elif id == HC.LOCAL_RATING_NUMERICAL: self._type = 'system:rating_numerical'
if id == HC.LOCAL_RATING_LIKE:
service_identifier = self._service_like.GetClientData( self._service_like.GetSelection() ).GetServiceIdentifier()
operator = '='
selection = self._value_like.GetSelection()
if selection == 0: value = '1'
elif selection == 1: value = '0'
elif selection == 2: value = 'rated'
elif selection == 3: value = 'not rated'
info = ( service_identifier, operator, value )
elif id == HC.LOCAL_RATING_NUMERICAL:
service = self._service_numerical.GetClientData( self._service_numerical.GetSelection() )
service_identifier = service.GetServiceIdentifier()
operator = self._sign_numerical.GetStringSelection()
if operator in ( '=rated', '=not rated', '=uncertain' ):
value = operator[1:]
operator = '='
else:
( lower, upper ) = service.GetExtraInfo()
value_raw = self._value_numerical.GetValue()
value = float( value_raw - lower ) / float( upper - lower )
info = ( service_identifier, operator, value )
elif self._type == HC.SYSTEM_PREDICATE_TYPE_RATIO: info = ( self._sign.GetStringSelection(), float( ( self._width.GetValue() ) / float( self._height.GetValue() ) ) )
elif self._type == HC.SYSTEM_PREDICATE_TYPE_SIZE: info = ( self._sign.GetStringSelection(), self._size.GetValue(), HC.ConvertUnitToInteger( self._unit.GetStringSelection() ) )
elif self._type == HC.SYSTEM_PREDICATE_TYPE_WIDTH: info = ( self._sign.GetStringSelection(), self._width.GetValue() )
elif self._type == HC.SYSTEM_PREDICATE_TYPE_SIMILAR_TO:
hex_filter = lambda c: c in '0123456789abcdef'
hash = filter( hex_filter, self._hash.GetValue() )
if len( hash ) == 0: hash == '00'
elif len( hash ) % 2 == 1: hash += '0' # since we are later decoding to byte
info = ( hash.decode( 'hex' ), self._max_hamming.GetValue() )
self._predicate = HC.Predicate( HC.PREDICATE_TYPE_SYSTEM, ( self._type, info ), None )
self.EndModal( wx.ID_OK )
@ -1562,62 +1678,7 @@ class DialogInputFileSystemPredicate( Dialog ):
except: pass
def GetString( self ):
if self._type == 'system:age': return 'system:age' + self._sign.GetStringSelection() + str( self._years.GetValue() ) + 'y' + str( self._months.GetValue() ) + 'm' + str( self._days.GetValue() ) + 'd'
elif self._type == 'system:duration': return 'system:duration' + self._sign.GetStringSelection() + str( self._duration_s.GetValue() * 1000 + self._duration_ms.GetValue() )
elif self._type == 'system:hash':
hex_filter = lambda c: c in '0123456789abcdef'
hash = filter( hex_filter, self._hash.GetValue() )
if len( hash ) == 0: hash == '00'
elif len( hash ) % 2 == 1: hash += '0' # since we are later decoding to byte
return 'system:hash=' + hash
elif self._type == 'system:height': return 'system:height' + self._sign.GetStringSelection() + str( self._height.GetValue() )
elif self._type == 'system:limit': return 'system:limit=' + str( self._limit.GetValue() )
elif self._type == 'system:mime': return 'system:mime=' + HC.mime_string_lookup[ self._mime_type.GetClientData( self._mime_type.GetSelection() ) ]
elif self._type == 'system:numtags': return 'system:numtags' + self._sign.GetStringSelection() + str( self._num_tags.GetValue() )
elif self._type == 'system:rating_like':
s = 'system:rating:' + self._service_like.GetClientData( self._service_like.GetSelection() ).GetServiceIdentifier().GetName() + '='
selection = self._value_like.GetSelection()
if selection == 0: s += '1'
elif selection == 1: s += '0'
elif selection == 2: s += 'rated'
elif selection == 3: s += 'not rated'
return s
elif self._type == 'system:rating_numerical':
service = self._service_numerical.GetClientData( self._service_numerical.GetSelection() )
s = 'system:rating:' + service.GetServiceIdentifier().GetName() + self._sign_numerical.GetStringSelection()
if self._sign_numerical.GetStringSelection() not in ( '=rated', '=not rated', '=uncertain' ):
( lower, upper ) = service.GetExtraInfo()
value = self._value_numerical.GetValue()
value_normalised = float( value - lower ) / float( upper - lower )
s += str( value_normalised )
return s
elif self._type == 'system:ratio': return 'system:ratio' + self._sign.GetStringSelection() + str( self._width.GetValue() ) + ':' + str( self._height.GetValue() )
elif self._type == 'system:size': return 'system:size' + self._sign.GetStringSelection() + str( self._size.GetValue() ) + self._unit.GetStringSelection()
elif self._type == 'system:width': return 'system:width' + self._sign.GetStringSelection() + str( self._width.GetValue() )
elif self._type == 'system:similar_to': return 'system:similar_to=' + self._hash.GetValue() + u'\u2248' + str( self._max_hamming.GetValue() )
def GetPredicate( self ): return self._predicate
class DialogInputMessageSystemPredicate( Dialog ):
@ -2347,7 +2408,8 @@ class DialogManageBoorus( Dialog ):
self._search_separator.Select( self._search_separator.FindString( search_separator ) )
self._search_separator.Bind( wx.EVT_CHOICE, self.EventHTML )
self._gallery_advance_num = wx.SpinCtrl( self._search_panel, min = 1, max = 1000, initial = gallery_advance_num )
self._gallery_advance_num = wx.SpinCtrl( self._search_panel, min = 1, max = 1000 )
self._gallery_advance_num.SetValue( gallery_advance_num )
self._gallery_advance_num.Bind( wx.EVT_SPIN, self.EventHTML )
self._thumb_classname = wx.TextCtrl( self._search_panel, value = thumb_classname )
@ -3630,7 +3692,8 @@ class DialogManageImageboards( Dialog ):
self._post_url = wx.TextCtrl( self._basic_info_panel, value = post_url )
self._flood_time = wx.SpinCtrl( self._basic_info_panel, min = 5, max = 1200, initial = flood_time )
self._flood_time = wx.SpinCtrl( self._basic_info_panel, min = 5, max = 1200 )
self._flood_time.SetValue( flood_time )
#
@ -4053,17 +4116,20 @@ class DialogManageOptionsLocal( Dialog ):
self._exclude_deleted_files = wx.CheckBox( self._file_page, label='' )
self._exclude_deleted_files.SetValue( self._options[ 'exclude_deleted_files' ] )
self._thumbnail_cache_size = wx.SpinCtrl( self._file_page, initial = int( self._options[ 'thumbnail_cache_size' ] / 1048576 ), min = 10, max = 3000 )
self._thumbnail_cache_size = wx.SpinCtrl( self._file_page, min = 10, max = 3000 )
self._thumbnail_cache_size.SetValue( int( self._options[ 'thumbnail_cache_size' ] / 1048576 ) )
self._thumbnail_cache_size.Bind( wx.EVT_SPINCTRL, self.EventThumbnailsUpdate )
self._estimated_number_thumbnails = wx.StaticText( self._file_page, label = '', style = wx.ST_NO_AUTORESIZE )
self._estimated_number_thumbnails = wx.StaticText( self._file_page, label = '' )
self._preview_cache_size = wx.SpinCtrl( self._file_page, initial = int( self._options[ 'preview_cache_size' ] / 1048576 ), min = 20, max = 3000 )
self._preview_cache_size = wx.SpinCtrl( self._file_page, min = 20, max = 3000 )
self._preview_cache_size.SetValue( int( self._options[ 'preview_cache_size' ] / 1048576 ) )
self._preview_cache_size.Bind( wx.EVT_SPINCTRL, self.EventPreviewsUpdate )
self._estimated_number_previews = wx.StaticText( self._file_page, label = '', style = wx.ST_NO_AUTORESIZE )
self._fullscreen_cache_size = wx.SpinCtrl( self._file_page, initial = int( self._options[ 'fullscreen_cache_size' ] / 1048576 ), min = 100, max = 3000 )
self._fullscreen_cache_size = wx.SpinCtrl( self._file_page, min = 100, max = 3000 )
self._fullscreen_cache_size.SetValue( int( self._options[ 'fullscreen_cache_size' ] / 1048576 ) )
self._fullscreen_cache_size.Bind( wx.EVT_SPINCTRL, self.EventFullscreensUpdate )
self._estimated_number_fullscreens = wx.StaticText( self._file_page, label = '', style = wx.ST_NO_AUTORESIZE )
@ -4078,7 +4144,8 @@ class DialogManageOptionsLocal( Dialog ):
self._thumbnail_height.SetValue( thumbnail_height )
self._thumbnail_height.Bind( wx.EVT_SPINCTRL, self.EventThumbnailsUpdate )
self._num_autocomplete_chars = wx.SpinCtrl( self._file_page, initial = self._options[ 'num_autocomplete_chars' ], min = 1, max = 100 )
self._num_autocomplete_chars = wx.SpinCtrl( self._file_page, min = 1, max = 100 )
self._num_autocomplete_chars.SetValue( self._options[ 'num_autocomplete_chars' ] )
self._num_autocomplete_chars.SetToolTipString( 'how many characters you enter before the gui fetches autocomplete results from the db' + os.linesep + 'increase this if you find autocomplete results are slow' )
self._listbook.AddPage( self._file_page, 'files and memory' )
@ -4130,28 +4197,35 @@ class DialogManageOptionsLocal( Dialog ):
self._file_system_predicate_age_sign = wx.Choice( self._file_system_predicates_page, choices=[ '<', u'\u2248', '>' ] )
self._file_system_predicate_age_sign.SetSelection( sign )
self._file_system_predicate_age_years = wx.SpinCtrl( self._file_system_predicates_page, initial = years, max = 30 )
self._file_system_predicate_age_months = wx.SpinCtrl( self._file_system_predicates_page, initial = months, max = 60 )
self._file_system_predicate_age_days = wx.SpinCtrl( self._file_system_predicates_page, initial = days, max = 90 )
self._file_system_predicate_age_years = wx.SpinCtrl( self._file_system_predicates_page, max = 30 )
self._file_system_predicate_age_years.SetValue( years )
self._file_system_predicate_age_months = wx.SpinCtrl( self._file_system_predicates_page, max = 60 )
self._file_system_predicate_age_months.SetValue( months )
self._file_system_predicate_age_days = wx.SpinCtrl( self._file_system_predicates_page, max = 90 )
self._file_system_predicate_age_days.SetValue( days )
( sign, s, ms ) = system_predicates[ 'duration' ]
self._file_system_predicate_duration_sign = wx.Choice( self._file_system_predicates_page, choices=[ '<', u'\u2248', '=', '>' ] )
self._file_system_predicate_duration_sign.SetSelection( sign )
self._file_system_predicate_duration_s = wx.SpinCtrl( self._file_system_predicates_page, initial = s, max = 3599 )
self._file_system_predicate_duration_ms = wx.SpinCtrl( self._file_system_predicates_page, initial = ms, max = 999 )
self._file_system_predicate_duration_s = wx.SpinCtrl( self._file_system_predicates_page, max = 3599 )
self._file_system_predicate_duration_s.SetValue( s )
self._file_system_predicate_duration_ms = wx.SpinCtrl( self._file_system_predicates_page, max = 999 )
self._file_system_predicate_duration_ms.SetValue( ms )
( sign, height ) = system_predicates[ 'height' ]
self._file_system_predicate_height_sign = wx.Choice( self._file_system_predicates_page, choices=[ '<', u'\u2248', '=', '>' ] )
self._file_system_predicate_height_sign.SetSelection( sign )
self._file_system_predicate_height = wx.SpinCtrl( self._file_system_predicates_page, initial = height, max = 200000 )
self._file_system_predicate_height = wx.SpinCtrl( self._file_system_predicates_page, max = 200000 )
self._file_system_predicate_height.SetValue( height )
limit = system_predicates[ 'limit' ]
self._file_system_predicate_limit = wx.SpinCtrl( self._file_system_predicates_page, initial = limit, max = 1000000 )
self._file_system_predicate_limit = wx.SpinCtrl( self._file_system_predicates_page, max = 1000000 )
self._file_system_predicate_limit.SetValue( limit )
( media, type ) = system_predicates[ 'mime' ]
@ -4170,14 +4244,16 @@ class DialogManageOptionsLocal( Dialog ):
self._file_system_predicate_num_tags_sign = wx.Choice( self._file_system_predicates_page, choices=[ '<', '=', '>' ] )
self._file_system_predicate_num_tags_sign.SetSelection( sign )
self._file_system_predicate_num_tags = wx.SpinCtrl( self._file_system_predicates_page, initial = num_tags, max = 2000 )
self._file_system_predicate_num_tags = wx.SpinCtrl( self._file_system_predicates_page, max = 2000 )
self._file_system_predicate_num_tags.SetValue( num_tags )
( sign, value ) = system_predicates[ 'local_rating_numerical' ]
self._file_system_predicate_local_rating_numerical_sign = wx.Choice( self._file_system_predicates_page, choices=[ '>', '<', '=', u'\u2248', '=rated', '=not rated', '=uncertain' ] )
self._file_system_predicate_local_rating_numerical_sign.SetSelection( sign )
self._file_system_predicate_local_rating_numerical_value = wx.SpinCtrl( self._file_system_predicates_page, initial = value, min = 0, max = 50000 )
self._file_system_predicate_local_rating_numerical_value = wx.SpinCtrl( self._file_system_predicates_page, min = 0, max = 50000 )
self._file_system_predicate_local_rating_numerical_value.SetValue( value )
value = system_predicates[ 'local_rating_like' ]
@ -4189,18 +4265,20 @@ class DialogManageOptionsLocal( Dialog ):
self._file_system_predicate_ratio_sign = wx.Choice( self._file_system_predicates_page, choices=[ '=', u'\u2248' ] )
self._file_system_predicate_ratio_sign.SetSelection( sign )
self._file_system_predicate_ratio_width = wx.SpinCtrl( self._file_system_predicates_page, initial = width, max = 50000 )
self._file_system_predicate_ratio_height = wx.SpinCtrl( self._file_system_predicates_page, initial = height, max = 50000 )
self._file_system_predicate_ratio_width = wx.SpinCtrl( self._file_system_predicates_page, max = 50000 )
self._file_system_predicate_ratio_width.SetValue( width )
self._file_system_predicate_ratio_height = wx.SpinCtrl( self._file_system_predicates_page, max = 50000 )
self._file_system_predicate_ratio_height.SetValue( height )
( sign, size, unit ) = system_predicates[ 'size' ]
self._file_system_predicate_size_sign = wx.Choice( self._file_system_predicates_page, choices=[ '<', u'\u2248', '=', '>' ] )
self._file_system_predicate_size_sign.SetSelection( sign )
self._file_system_predicate_size = wx.SpinCtrl( self._file_system_predicates_page, initial = size, max = 1048576 )
self._file_system_predicate_size = wx.SpinCtrl( self._file_system_predicates_page, max = 1048576 )
self._file_system_predicate_size.SetValue( size )
self._file_system_predicate_size_unit = wx.Choice( self._file_system_predicates_page, choices=[ 'b', 'B', 'Kb', 'KB', 'Mb', 'MB', 'Gb', 'GB' ] )
self._file_system_predicate_size_unit = wx.Choice( self._file_system_predicates_page, choices=[ 'B', 'KB', 'MB', 'GB' ] )
self._file_system_predicate_size_unit.SetSelection( unit )
( sign, width ) = system_predicates[ 'width' ]
@ -4208,7 +4286,16 @@ class DialogManageOptionsLocal( Dialog ):
self._file_system_predicate_width_sign = wx.Choice( self._file_system_predicates_page, choices=[ '<', u'\u2248', '=', '>' ] )
self._file_system_predicate_width_sign.SetSelection( sign )
self._file_system_predicate_width = wx.SpinCtrl( self._file_system_predicates_page, initial = width, max = 200000 )
self._file_system_predicate_width = wx.SpinCtrl( self._file_system_predicates_page, max = 200000 )
self._file_system_predicate_width.SetValue( width )
( sign, num_words ) = system_predicates[ 'num_words' ]
self._file_system_predicate_num_words_sign = wx.Choice( self._file_system_predicates_page, choices=[ '<', u'\u2248', '=', '>' ] )
self._file_system_predicate_num_words_sign.SetSelection( sign )
self._file_system_predicate_num_words = wx.SpinCtrl( self._file_system_predicates_page, max = 1000000 )
self._file_system_predicate_num_words.SetValue( num_words )
self._listbook.AddPage( self._file_system_predicates_page, 'default file system predicates' )
@ -4403,7 +4490,7 @@ class DialogManageOptionsLocal( Dialog ):
hbox = wx.BoxSizer( wx.HORIZONTAL )
hbox.AddF( wx.StaticText( self._file_system_predicates_page, label='system:numtags' ), FLAGS_MIXED )
hbox.AddF( wx.StaticText( self._file_system_predicates_page, label='system:num_tags' ), FLAGS_MIXED )
hbox.AddF( self._file_system_predicate_num_tags_sign, FLAGS_MIXED )
hbox.AddF( self._file_system_predicate_num_tags, FLAGS_MIXED )
@ -4451,6 +4538,14 @@ class DialogManageOptionsLocal( Dialog ):
vbox.AddF( hbox, FLAGS_EXPAND_SIZER_PERPENDICULAR )
hbox = wx.BoxSizer( wx.HORIZONTAL )
hbox.AddF( wx.StaticText( self._file_system_predicates_page, label='system:num_words' ), FLAGS_MIXED )
hbox.AddF( self._file_system_predicate_num_words_sign, FLAGS_MIXED )
hbox.AddF( self._file_system_predicate_num_words, FLAGS_MIXED )
vbox.AddF( hbox, FLAGS_EXPAND_SIZER_PERPENDICULAR )
self._file_system_predicates_page.SetSizer( vbox )
#
@ -4682,6 +4777,7 @@ class DialogManageOptionsLocal( Dialog ):
system_predicates[ 'ratio' ] = ( self._file_system_predicate_ratio_sign.GetSelection(), self._file_system_predicate_ratio_width.GetValue(), self._file_system_predicate_ratio_height.GetValue() )
system_predicates[ 'size' ] = ( self._file_system_predicate_size_sign.GetSelection(), self._file_system_predicate_size.GetValue(), self._file_system_predicate_size_unit.GetSelection() )
system_predicates[ 'width' ] = ( self._file_system_predicate_width_sign.GetSelection(), self._file_system_predicate_width.GetValue() )
system_predicates[ 'num_words' ] = ( self._file_system_predicate_num_words_sign.GetSelection(), self._file_system_predicate_num_words.GetValue() )
self._options[ 'file_system_predicates' ] = system_predicates

41
include/ClientGUIManagement.py
View File

@ -3535,36 +3535,25 @@ class ManagementPanelQuery( ManagementPanel ):
if predicate is not None:
if predicate in ( 'system:size', 'system:age', 'system:hash', 'system:limit', 'system:numtags', 'system:width', 'system:height', 'system:ratio', 'system:duration', u'system:mime', u'system:rating', u'system:similar_to' ):
( predicate_type, value ) = predicate.GetInfo()
if predicate_type == HC.PREDICATE_TYPE_SYSTEM:
with ClientGUIDialogs.DialogInputFileSystemPredicate( self, predicate ) as dlg:
if dlg.ShowModal() == wx.ID_OK: predicate = dlg.GetString()
else: return
( system_predicate_type, info ) = value
if system_predicate_type in [ HC.SYSTEM_PREDICATE_TYPE_NUM_TAGS, HC.SYSTEM_PREDICATE_TYPE_LIMIT, HC.SYSTEM_PREDICATE_TYPE_SIZE, HC.SYSTEM_PREDICATE_TYPE_AGE, HC.SYSTEM_PREDICATE_TYPE_HASH, HC.SYSTEM_PREDICATE_TYPE_WIDTH, HC.SYSTEM_PREDICATE_TYPE_HEIGHT, HC.SYSTEM_PREDICATE_TYPE_RATIO, HC.SYSTEM_PREDICATE_TYPE_DURATION, HC.SYSTEM_PREDICATE_TYPE_NUM_WORDS, HC.SYSTEM_PREDICATE_TYPE_MIME, HC.SYSTEM_PREDICATE_TYPE_RATING, HC.SYSTEM_PREDICATE_TYPE_SIMILAR_TO ]:
with ClientGUIDialogs.DialogInputFileSystemPredicate( self, system_predicate_type ) as dlg:
if dlg.ShowModal() == wx.ID_OK: predicate = dlg.GetPredicate()
else: return
elif system_predicate_type == HC.SYSTEM_PREDICATE_TYPE_UNTAGGED: predicate = HC.Predicate( HC.PREDICATE_TYPE_SYSTEM, ( HC.SYSTEM_PREDICATE_TYPE_NUM_TAGS, ( '=', 0 ) ), None )
elif predicate == 'system:untagged': predicate = 'system:numtags=0'
if self._current_predicates_box.HasPredicate( predicate ): self._current_predicates_box.RemovePredicate( predicate )
else:
if predicate in ( 'system:inbox', 'system:archive', 'system:local', 'system:not local' ):
if predicate == 'system:inbox': removee = 'system:archive'
elif predicate == 'system:archive': removee = 'system:inbox'
elif predicate == 'system:local': removee = 'system:not local'
elif predicate == 'system:not local': removee = 'system:local'
else:
if predicate.startswith( '-' ): removee = predicate[1:]
else: removee = '-' + predicate
if self._current_predicates_box.HasPredicate( removee ): self._current_predicates_box.RemovePredicate( removee )
self._current_predicates_box.AddPredicate( predicate )
else: self._current_predicates_box.AddPredicate( predicate )
self._DoQuery()

26
include/ClientGUIMedia.py
View File

@ -565,6 +565,24 @@ class MediaPanel( ClientGUIMixins.ListeningMediaList, wx.ScrolledWindow ):
HC.pubsub.pub( 'focus_changed', self._page_key, media )
def _ShowSelectionInNewQueryPage( self ):
hashes = self._GetSelectedHashes()
if hashes is not None and len( hashes ) > 0:
search_context = CC.FileSearchContext()
unsorted_file_query_result = wx.GetApp().Read( 'media_results', search_context, hashes )
hashes_to_media_results = { media_result.GetHash() : media_result for media_result in unsorted_file_query_result }
sorted_media_results = [ hashes_to_media_results[ hash ] for hash in hashes ]
HC.pubsub.pub( 'new_page_query', self._file_service_identifier, initial_media_results = sorted_media_results )
def _UploadFiles( self, file_service_identifier ):
hashes = self._GetSelectedHashes( not_uploaded_to = file_service_identifier )
@ -1096,6 +1114,7 @@ class MediaPanelThumbnails( MediaPanel ):
elif command == 'scroll_end': self._ScrollEnd()
elif command == 'scroll_home': self._ScrollHome()
elif command == 'select_all': self._SelectAll()
elif command == 'show_selection_in_new_query_page': self._ShowSelectionInNewQueryPage()
elif command == 'upload': self._UploadFiles( data )
elif command == 'key_up': self._MoveFocussedThumbnail( -1, 0, False )
elif command == 'key_down': self._MoveFocussedThumbnail( 1, 0, False )
@ -1421,6 +1440,13 @@ class MediaPanelThumbnails( MediaPanel ):
menu.AppendSeparator()
if multiple_selected:
menu.Append( CC.MENU_EVENT_ID_TO_ACTION_CACHE.GetId( 'show_selection_in_new_query_page' ), 'open selection in a new page' )
menu.AppendSeparator()
copy_menu = wx.Menu()
copy_menu.Append( CC.MENU_EVENT_ID_TO_ACTION_CACHE.GetId( 'copy_files' ), copy_phrase )

8
include/ClientGUIMixins.py
View File

@ -107,17 +107,17 @@ class MediaList():
singletons = set()
keys_to_multiples_media = collections.defaultdict( list )
keys_to_medias = collections.defaultdict( list )
for media in self._singleton_media:
key = media.GetTags().GetNamespaceSlice( collect_by )
keys_to_multiples_media[ key ].append( media )
keys_to_medias[ key ].append( media )
self._singleton_media = singletons
self._collected_media = set( [ self._GenerateMediaCollection( [ media.GetMediaResult() for media in multiples_media ] ) for multiples_media in keys_to_multiples_media.values() ] )
self._singleton_media = set( [ medias[0] for medias in keys_to_medias.values() if len( medias ) == 1 ] )
self._collected_media = set( [ self._GenerateMediaCollection( [ media.GetMediaResult() for media in medias ] ) for medias in keys_to_medias.values() if len( medias ) > 1 ] )
self._sorted_media = list( self._singleton_media ) + list( self._collected_media )

8
include/ClientGUIPages.py
View File

@ -351,8 +351,9 @@ class PagePetitions( PageWithMedia ):
class PageQuery( PageWithMedia ):
def __init__( self, parent, file_service_identifier, initial_predicates = [] ):
def __init__( self, parent, file_service_identifier, initial_media_results = [], initial_predicates = [] ):
self._initial_media_results = initial_media_results
self._initial_predicates = initial_predicates
PageWithMedia.__init__( self, parent, file_service_identifier )
@ -360,7 +361,10 @@ class PageQuery( PageWithMedia ):
def _InitManagementPanel( self ): self._management_panel = ClientGUIManagement.ManagementPanelQuery( self._search_preview_split, self, self._page_key, self._file_service_identifier, initial_predicates = self._initial_predicates )
def _InitMediaPanel( self ): self._media_panel = ClientGUIMedia.MediaPanelNoQuery( self, self._page_key, self._file_service_identifier )
def _InitMediaPanel( self ):
if len( self._initial_media_results ) == 0: self._media_panel = ClientGUIMedia.MediaPanelNoQuery( self, self._page_key, self._file_service_identifier )
else: self._media_panel = ClientGUIMedia.MediaPanelThumbnails( self, self._page_key, self._file_service_identifier, self._initial_predicates, self._initial_media_results )
class PageThreadDumper( PageWithMedia ):

235
include/HydrusConstants.py
View File

@ -12,8 +12,6 @@ import traceback
import wx
import yaml
locale.setlocale( locale.LC_ALL, '' )
BASE_DIR = sys.path[0]
DB_DIR = BASE_DIR + os.path.sep + 'db'
@ -30,7 +28,7 @@ TEMP_DIR = BASE_DIR + os.path.sep + 'temp'
# Misc
NETWORK_VERSION = 9
SOFTWARE_VERSION = 62
SOFTWARE_VERSION = 63
UNSCALED_THUMBNAIL_DIMENSIONS = ( 200, 200 )
@ -232,6 +230,31 @@ header_and_mime = [
( '%PDF', APPLICATION_PDF )
]
PREDICATE_TYPE_SYSTEM = 0
PREDICATE_TYPE_TAG = 1
PREDICATE_TYPE_NAMESPACE = 2
SYSTEM_PREDICATE_TYPE_EVERYTHING = 0
SYSTEM_PREDICATE_TYPE_INBOX = 1
SYSTEM_PREDICATE_TYPE_ARCHIVE = 2
SYSTEM_PREDICATE_TYPE_UNTAGGED = 3
SYSTEM_PREDICATE_TYPE_NUM_TAGS = 4
SYSTEM_PREDICATE_TYPE_LIMIT = 5
SYSTEM_PREDICATE_TYPE_SIZE = 6
SYSTEM_PREDICATE_TYPE_AGE = 7
SYSTEM_PREDICATE_TYPE_HASH = 8
SYSTEM_PREDICATE_TYPE_WIDTH = 9
SYSTEM_PREDICATE_TYPE_HEIGHT = 10
SYSTEM_PREDICATE_TYPE_RATIO = 11
SYSTEM_PREDICATE_TYPE_DURATION = 12
SYSTEM_PREDICATE_TYPE_MIME = 13
SYSTEM_PREDICATE_TYPE_RATING = 14
SYSTEM_PREDICATE_TYPE_SIMILAR_TO = 15
SYSTEM_PREDICATE_TYPE_NOT_UPLOADED_TO = 16
SYSTEM_PREDICATE_TYPE_LOCAL = 17
SYSTEM_PREDICATE_TYPE_NOT_LOCAL = 18
SYSTEM_PREDICATE_TYPE_NUM_WORDS = 19
wxk_code_string_lookup = {
wx.WXK_SPACE: 'space',
wx.WXK_BACK: 'backspace',
@ -287,7 +310,9 @@ wxk_code_string_lookup = {
wx.WXK_NUMPAD_ADD: 'numpad +',
wx.WXK_NUMPAD_DIVIDE: 'numpad /',
wx.WXK_NUMPAD_SUBTRACT: 'numpad -',
wx.WXK_NUMPAD_MULTIPLY: 'numpad *'
wx.WXK_NUMPAD_MULTIPLY: 'numpad *',
wx.WXK_NUMPAD_DELETE: 'numpad delete',
wx.WXK_NUMPAD_DECIMAL: 'numpad decimal'
}
# request checking
@ -788,6 +813,20 @@ def ConvertTimeToPrettyTime( secs ):
return time.strftime( '%H:%M:%S', time.gmtime( secs ) )
def ConvertUnitToInteger( unit ):
if unit == 'B': return 8
elif unit == 'KB': return 1024
elif unit == 'MB': return 1048576
elif unit == 'GB': return 1073741824
def ConvertUnitToString( unit ):
if unit == 8: return 'B'
elif unit == 1024: return 'KB'
elif unit == 1048576: return 'MB'
elif unit == 1073741824: return 'GB'
def ConvertZoomToPercentage( zoom ):
zoom = zoom * 100.0
@ -881,6 +920,18 @@ def IsCollection( mime ): return mime in ( APPLICATION_HYDRUS_CLIENT_COLLECTION,
def IsImage( mime ): return mime in ( IMAGE_JPEG, IMAGE_GIF, IMAGE_PNG, IMAGE_BMP )
def SearchEntryMatchesPredicate( search_entry, predicate ):
( predicate_type, info ) = predicate.GetInfo()
if predicate_type == PREDICATE_TYPE_TAG:
( operator, value ) = info
return SearchEntryMatchesTag( search_entry, value )
else: return False
def SearchEntryMatchesTag( search_entry, tag ):
# note that at no point is the namespace checked against the search_entry!
@ -893,7 +944,6 @@ def SearchEntryMatchesTag( search_entry, tag ):
else: return tag.startswith( search_entry )
def SplayListForDB( xs ): return '(' + ','.join( [ '"' + str( x ) + '"' for x in xs ] ) + ')'
def SplayTupleListForDB( first_column_name, second_column_name, xys ): return ' OR '.join( [ '( ' + first_column_name + '=' + str( x ) + ' AND ' + second_column_name + ' IN ' + SplayListForDB( ys ) + ' )' for ( x, ys ) in xys ] )
@ -1533,6 +1583,181 @@ class JobServer():
self._result_ready.set()
class Predicate():
def __init__( self, predicate_type, value, count ):
self._predicate_type = predicate_type
self._value = value
self._count = count
def __eq__( self, other ): return self.__hash__() == other.__hash__()
def __hash__( self ): return ( self._predicate_type, self._value ).__hash__()
def __ne__( self, other ): return self.__hash__() != other.__hash__()
def GetCountlessCopy( self ): return Predicate( self._predicate_type, self._value, None )
def GetCount( self ): return self._count
def GetInfo( self ): return ( self._predicate_type, self._value )
def GetPredicateType( self ): return self._predicate_type
def GetUnicode( self ):
if self._predicate_type == PREDICATE_TYPE_SYSTEM:
( system_predicate_type, info ) = self._value
if system_predicate_type == SYSTEM_PREDICATE_TYPE_EVERYTHING: base = u'system:everything'
elif system_predicate_type == SYSTEM_PREDICATE_TYPE_INBOX: base = u'system:inbox'
elif system_predicate_type == SYSTEM_PREDICATE_TYPE_ARCHIVE: base = u'system:archive'
elif system_predicate_type == SYSTEM_PREDICATE_TYPE_UNTAGGED: base = u'system:untagged'
elif system_predicate_type == SYSTEM_PREDICATE_TYPE_LOCAL: base = u'system:local'
elif system_predicate_type == SYSTEM_PREDICATE_TYPE_NOT_LOCAL: base = u'system:not_local'
elif system_predicate_type in ( SYSTEM_PREDICATE_TYPE_NUM_TAGS, SYSTEM_PREDICATE_TYPE_WIDTH, SYSTEM_PREDICATE_TYPE_HEIGHT, SYSTEM_PREDICATE_TYPE_RATIO, SYSTEM_PREDICATE_TYPE_DURATION, SYSTEM_PREDICATE_TYPE_NUM_WORDS ):
if system_predicate_type == SYSTEM_PREDICATE_TYPE_NUM_TAGS: base = u'system:num_tags'
elif system_predicate_type == SYSTEM_PREDICATE_TYPE_WIDTH: base = u'system:width'
elif system_predicate_type == SYSTEM_PREDICATE_TYPE_HEIGHT: base = u'system:height'
elif system_predicate_type == SYSTEM_PREDICATE_TYPE_RATIO: base = u'system:ratio'
elif system_predicate_type == SYSTEM_PREDICATE_TYPE_DURATION: base = u'system:duration'
elif system_predicate_type == SYSTEM_PREDICATE_TYPE_NUM_WORDS: base = u'system:num_words'
if info is not None:
( operator, value ) = info
base = base + operator + unicode( value )
elif system_predicate_type == SYSTEM_PREDICATE_TYPE_SIZE:
base = u'system:size'
if info is not None:
( operator, size, unit ) = info
base = base + operator + unicode( size ) + ConvertUnitToString( unit )
elif system_predicate_type == SYSTEM_PREDICATE_TYPE_LIMIT:
base = u'system:limit'
if info is not None:
value = info
base = base + u'=' + unicode( value )
elif system_predicate_type == SYSTEM_PREDICATE_TYPE_AGE:
base = u'system:age'
if info is not None:
( operator, years, months, days ) = info
base = base + operator + unicode( years ) + u'y' + unicode( months ) + u'm' + unicode( days ) + u'd'
elif system_predicate_type == SYSTEM_PREDICATE_TYPE_HASH:
base = u'system:hash'
if info is not None:
hash = info
base = base + u'=' + hash.encode( 'hex' )
elif system_predicate_type == SYSTEM_PREDICATE_TYPE_MIME:
base = u'system:mime'
if info is not None:
mime = info
base = base + u'=' + mime_string_lookup[ mime ]
elif system_predicate_type == SYSTEM_PREDICATE_TYPE_RATING:
base = u'system:rating'
if info is not None:
( service_identifier, operator, value ) = info
base = base + u':' + service_identifier.GetName() + operator + unicode( value )
elif system_predicate_type == SYSTEM_PREDICATE_TYPE_SIMILAR_TO:
base = u'system:similar_to'
if info is not None:
( hash, max_hamming ) = info
base = base + u'=' + hash.encode( 'hex' ) + u'\u2248' + unicode( max_hamming )
elif system_predicate_type == SYSTEM_PREDICATE_TYPE_NOT_UPLOADED_TO:
base = u'system:not_uploaded_to'
if info is not None:
service_identifier = info
base = base + u':' + service_identifier.GetName()
elif self._predicate_type == PREDICATE_TYPE_TAG:
( operator, tag ) = self._value
if operator == '-': base = u'-'
elif operator == '+': base = u''
base = base + tag
elif self._predicate_type == PREDICATE_TYPE_NAMESPACE:
( operator, namespace ) = self._value
if operator == '-': base = u'-'
elif operator == '+': base = u''
base = base + namespace + u':*'
if self._count is None: return base
else: return base + u' (' + ConvertIntToPrettyString( self._count ) + u')'
def GetValue( self ): return self._value
def SetOperator( self, operator ):
if self._predicate_type == PREDICATE_TYPE_TAG:
( old_operator, tag ) = self._value
self._value = ( operator, tag )
class ResponseContext():
def __init__( self, status_code, mime = None, body = '', filename = None, cookies = [] ):

6
include/HydrusImageHandling.py
View File

@ -70,6 +70,8 @@ def GenerateAnimatedFrame( pil_image, target_resolution, canvas ):
if pil_image.mode == 'P' and 'transparency' in pil_image.info:
# I think gif problems are around here somewhere; the transparency info is not converted to RGBA properly, so it starts drawing colours when it should draw nothing
current_frame = current_frame.convert( 'RGBA' )
if canvas is None: canvas = current_frame
@ -87,8 +89,6 @@ def GenerateHydrusBitmapFromFile( file ):
def GenerateHydrusBitmapFromPILImage( pil_image ):
( image_width, image_height ) = pil_image.size
if pil_image.mode == 'RGBA' or ( pil_image.mode == 'P' and pil_image.info.has_key( 'transparency' ) ):
if pil_image.mode == 'P': pil_image = pil_image.convert( 'RGBA' )
@ -97,7 +97,7 @@ def GenerateHydrusBitmapFromPILImage( pil_image ):
else:
if pil_image.mode in ( 'P', 'L', 'LA' ): pil_image = pil_image.convert( 'RGB' )
if pil_image.mode != 'RGB': pil_image = pil_image.convert( 'RGB' )
return HydrusBitmap( pil_image.tostring(), wx.BitmapBufferFormat_RGB, pil_image.size )

9
server.pyw
View File

@ -4,6 +4,15 @@
# To Public License, Version 2, as published by Sam Hocevar. See
# http://sam.zoy.org/wtfpl/COPYING for more details.
import string
string.whitespace
# this is some woo woo - if you call it after the locale, it has 0xa0 (non-breaking space) (non-ascii!!) included
# if you call it before, the locale call doesn't update
# what a mess!
import locale
locale.setlocale( locale.LC_ALL, '' )
import os
from include import HydrusConstants as HC
from include import ServerController