RepoMirrors
/
hydrus
mirror of https://github.com/hydrusnetwork/hydrus.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Version 534 

closes #1373, closes #343
This commit is contained in:
Hydrus Network Developer 2023-07-05 15:52:58 -05:00
parent c4bf014bde
commit 623e430ded
No known key found for this signature in database
GPG Key ID: 76249F053212133C
37 changed files with 1130 additions and 265 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

107
docs/changelog.md
View File

@ -7,6 +7,56 @@ title: Changelog
!!! note
This is the new changelog, only the most recent builds. For all versions, see the [old changelog](old_changelog.html).
## [Version 534](https://github.com/hydrusnetwork/hydrus/releases/tag/v534)
### user submissions
* thanks to a user, we now have SAI2 (.sai2) file support!
* thanks to a user, the duplicate filter now says if one file has audio. this complements the recent Hydrus Video Deduplicator (https://github.com/appleappleapplenanner/hydrus-video-deduplicator), which can queue videos up in your dupe filter
* thanks to a user, we now have some nice svg images in the help->links(?) menu instead of gritty bitmaps
* thanks to a user, some help documentation for recent client vs hydrus_client changes got fixed
### quality of life/new stuff
* the media viewer's top-area 'removed from x' lines for files deleted from a local file service no longer appear--unless that file is currently in the trash. on clients with busy multiple local file services, they were mostly just annoying and spammy. if you need this data, hit up the right-click menu of the file--it is still listed there
* the 'loading' media page now draws a background in the same colour as the thumbnail grid, so new searches or refreshes will no longer flash to a default grey colour--it should just be a smooth thumbs gone/thumbs back now
* added a new shortcut action, 'copy small bmp of image for quick source lookups', for last week's new bitmap copy action
* it turns out PNG and WEBP files can have EXIF data, and our existing scanner works with them, so the EXIF scanner now looks at PNGs and WEBPs too. PNGs appear to be rare, about 1-in-200. I will retroactively scan your existing WEBPs, since they have EXIF more commonly, maybe as high as 1-in-5, and are less common as a filetype anyway so the scan will be less work, but when you update you will get a yes/no dialog asking if you want to do PNGs too. it isn't a big deal and you can always queue it up later if you want
### fixes
* I banged my head against the notes layout code and actually had great success--a variety of borked note-spilling-over-into-nothing and note-stretching-itself-crazy and note-has-fifty-pixels-of-margin issues are fixed. let me know if you still have crazy notes anywhere
* the duplicate filter right-hand hover is now more aggressive about getting out of the way of the notes hover, especially when the notes hover jitter-resizes itself a few extra pixels of height. the notes hover should no longer ever overlap the duplicate filter hover's top buttons--very sorry this took so long
* when you drag and drop thumbnails out of the program while using an automatic pattern to rename them (_options->gui_), all the filenames are now unique, adding '(x)' after their name as needed for dedupe. previously, on duplicates, it was basically doing a weird spam-merge
* fixed an issue when sanitizing export filenames when the destination directory's file system type cannot be determined
* fixed a bug when doing a search in a deleted file domain with 'file import time' as the file sort
* fixed a bug when hitting the shortcut for 'open file in media viewer' on a non-local file
* fixed a bug when the client wants to figure out view options for a file that has mime 'application/unknown'
* I may have improved the 'woah the db caches are unsynced' error handling in the 'what siblings and parents does this have?' check when you right-click tags
### weird bitmap pastes
* fixed the new 'paste image' button under `system:similar files` for a certain category of unusual clipboard bitmaps, including several that hydrus itself generates, where it turns out the QImage storage system stores extra padding bytes on each line of pixels
* fixed the new 'paste image' button when the incoming bitmap has a useless alpha channel (e.g. 100% transparent). this was not being stripped like it is for imported images, and so some similar files data was not lining up
* many bitmaps copied from other programs like Firefox remain slightly different to what hydrus generates (even though both are at 100% scale). my best guess here is that there is some differing ICC-profile-like colour adjustment happening somewhere, probably either a global browser setting, the browser obeying a global GPU setting, a simply better application of such image metadata on the browser's side, or maybe a stripping of such data, since it seems a 'copy image' event in Firefox also generates and attaches to your clipboard a temporary png file in your temp folder, so maybe the bitmap that we pull from the clipboard is actually generated during some conversion process amidst all that, and it loses some jpeg colour data. whatever the case here, it changes the pixel hash and subtly alters the perceptual hash in many cases. I'm bumping the default distance on this search predicate up to 8 now, to catch the weirder instances
### misc
* the 'does the db partition have 500MB free?' check that runs on database boot now occurs after some initial database cleanup, and it will use half the total database size instead, if that is smaller than 500MB, down to 64MB (issue #1373)
* added a note to the 'running from source' help that the newer mpv dll seems to work on Qt5 and Windows 7 (issue #1338)
* the twitter parsers and gugs are removed from the defaults for new users. a shame, but we'll see what happens in future
* more misc linting cleanup
### ratings on the client api
* the services object now shows `star_shape` and `min_stars` and `max_stars` for like/dislike and numerical rating services
* the file metadata object now has a 'ratings' key, which lists `rating_service_key->rating` for all the client's rating services. this thing is simple and uses human-friendly values, but it can hold several different data types, so check the help for details and examples
* a new permission, 'edit ratings', is added.
* a new command, `/edit_ratings/set_rating`, is added. Guess what it does! (issue #343)
* the help is updated for these
* the unit tests are updated for these
* the client api version is now 48
## [Version 533](https://github.com/hydrusnetwork/hydrus/releases/tag/v533)
### macOS App crashes
@ -307,60 +357,3 @@ title: Changelog
* the 'API URL' system for url classes now supports File URLs--this may help you figure out some CDN redirects and similar. in a special rule for these File URLs, both URLs will be associated with the imported file (normally, Post API URLs are not saved as Known URLs). relatedly, I have renamed this system broadly to 'api/redirect url', since we use it for a bunch of non-API stuff now
* fixed a problem where deleting one of the new inc/dec rating services was not clearing the actual number ratings for that service from the database, causing service-id error hell on loading files with those orphaned rating records. sorry for the trouble, this slipped through testing! any users who were affected by this will also be fixed (orphan records cleared out) on update (issue #1357)
* the client cleans up the temporary paths used by file imports more carefully now: it tries more times to delete 'sticky' temp files; it tries to clear them again immediately on shutdown; and it stores them all in the hydrus temp subdirectory where they are less loose and will be captured by the final directory clear on shutdown (issue #1356)
## [Version 524](https://github.com/hydrusnetwork/hydrus/releases/tag/v524)
### timestamp sidecars
* the sidecars system now supports timestamps. it just uses the unix timestamp number, but if you need it, you can use string conversion to create a full datestring. each sidecar node only selects/sets that one timestamp, so this may get spammy if you want to migrate everything, but you can now migrate archived/imported/whatever time from one client to another! the content updates from sidecar imports apply immediately _after_ the file is fully imported, so it is safe and good to sidecar-import 'my files imported time' etc.. for new files, and it should all get set correctly, but obviously let me know otherwise. if you set 'archived time', the files have to be in an archived state immediately after import, which means importing and archiving them previously, or hitting 'archive all imports' on the respective file import options
* sidecars are getting complex, so I expect I will soon add a button that sets up a 'full' JSON sidecar import/export in one click, basically just spamming/sucking everything the sidecar system can do, pretty soon, so it is easier to set up larger migrations
### timestamp merge
* the duplicate merge options now have an action for 'sync file modified date?'. you can set so both files get their earliest (the new default for 'they are the same'), or that the earlier worse can be applied to the later better (the new default for 'this is better') (issue #1203)
* in the duplicate system, when URLs are merged, their respective domain-based timestamps are also merged according to the earliest, as above
### more timestamps
* hydrus now supports timestamps before 1970. should be good now, lol, back to 1AD (and my tests show BC dates seem to be working too?). it is probably a meme to apply a modified date of 1505 to some painting, but when I add timestamps to the API maybe we can have some fun. btw calendar calculations and timezones are hell on earth at times, and there's a decent chance that your pre-1970 dates may show up on hour out of phase in labels (a daylight savings time thing) of what you enter in some other area of UI. in either case, my code is not clever enough to apply DST schedules retroactively to older dates, so your search ranges may simply be an hour out back in 1953. it sounds stupid, but it may matter if we are talking midnight boundaries, so let me know how you find it
* when you set a new file modified date, the file on disk's modified date will only be updated if the date set is after 1980-01-01 (Windows) or 1970-01-01 (Linux) due to system limitations
* fixed a typo bug in last week's work that meant file service timestamp editing was not updating the media object (i.e. changes were not visible until a restart)
* fixed a bug where collections that contained files with delete timestamps were throwing errors on display. (they were calculating aggregate timestamp data wrong)
* I rejiggered how the 'is this timestamp sensible?' test applies. this test essentially discounts any timestamp before 1970-01-08 to catch any weird mis-parses and stop them nuking your aggregate modified timestamp values. it now won't apply to internal duplicate merge and so on, but it still applies when you parse timestamps in the downloader system, so you still can't parse anything pre-1970 for now
* one thing I noticed is my '5 years 1 months ago' calculation, which uses a fixed 30 day month and doesn't count the extra day of leap years, is showing obviously increasingly inaccurate numbers here. I'll fix it up
### export folders
* export folders can now show a popup while they work. there's a new checkbox for it in their edit UI. default is ON, so you'll start seeing popups for export folders that run in the background. this popup is cancellable, too, so you can now stop in-progress export runs if things seem wrong
* both import and export folders will force-show working popups whenever you trigger them manually
* export folders no longer have the weird and confusing 'paused' and 'run regularly?' duality. this was a legacy error handling thing, now cleaned up and merged into 'run regularly?'
* when 'run regularly?' is unchecked, the run period and new 'show popup while working regularly?' checkboxes are now disabled
### misc
* added 'system:ratio is square/portrait/landscape' nicer label aliases for =/taller/wider 1:1 ratio. I added them to the quick-select list on the edit panel, too. they also parse in the system predicate parser!
* I added a bit to the 'getting started with downloading' help page about getting access to difficult sites. I refer to Hydrus Companion as a good internal login solution, and link to yt-dlp, gallery-dl, and imgbrd-grabber with a little discussion on setting up external import workflows. I tried gallery-dl on twitter this week and it was excellent. it can also take your login credentials as either user/pass or cookies.txt (or pull cookies straight from firefox/safari) and give access to nsfw. since twitter has rapidly become a pain for us recently, I will be pointing people to gallery-dl for now
* fixed my Qt subclass definitions for PySide6 6.5.0, which strictly requires the Qt object to be the rightmost base class in multiple inheritance subclasses, wew. this his AUR users last week, I understand!
### client api (and local booru lol)
* if you set the Client API to not allow non-local connections, it now binds to 127.0.0.1 and ::1 specifically, which tell your OS we only want the loopback interface. this increases security, and on Windows _should_ mean it only does that first-time firewall dialog popup when 'allow non-local connections' is unchecked
* I brushed up the manage services UI for the Client API. the widgets all line up better now, and turning the service on and off isn't the awkward '[] do not run the service' any more
* fixed the 'disable idle mode if the client api does stuff' check, which was wired up wrong! also, the reset here now fires as a request starts, not when it is complete, meaning if you are already in idle mode, a client api request will now quickly cancel idle mode and hopefully free up any locked database situation promptly
### boring cleanup and stuff
* reworked all timestamp-datetime conversion to be happier with pre-1970 dates regardless of system/python support. it is broadly improved all around
* refactored all of the HydrusData time functions and much of ClientTime to a new HydrusTime module
* refactored the ClientData time stuff to ClientTime
* refactored some thread/process functions from HydrusData to HydrusThreading
* refactored some list splitting/throttling functions from HydrusData to a new HydrusLists module
* refactored the file filter out of ClientMedia and into the new ClientMediaFileFilter, and reworked things so the medialist filter jobs now happen at the filter level. this was probably done the wrong way around, but oh well
* expanded the new TimestampData object a bit, it can now give a nice descriptive string of itself
* wrote a new widget to edit TimestampData stubs
* wrote some unit tests for the new timestamp sidecar importer and exporter
* updated my multi-column list system to handle the deprecation of a column definition (today it was the 'paused' column in manage export folders list)
* it should also be able to handle new column definitions appearing
* fixed an error popup that still said 'run repair invalid tags' instead of 'run fix invalid tags'
* the FILE_SERVICES constant now holds the 'all deleted files' virtual domain. this domain keeps slipping my logic, so fingers crossed this helps. also means you can select it in 'system:file service' and stuff now
* misc cleaning and linting work

114
docs/developer_api.md
View File

@ -218,7 +218,7 @@ When it does this, it gives you this structure, typically under a `services` key
},
"616c6c206c6f63616c206d65646961" : {
"name" : "all my files",
"ype" : 21,
"type" : 21,
"type_pretty" : "virtual combined local media service"
},
"616c6c206b6e6f776e2066696c6573" : {
@ -234,12 +234,16 @@ When it does this, it gives you this structure, typically under a `services` key
"74d52c6238d25f846d579174c11856b1aaccdb04a185cb2c79f0d0e499284f2c" : {
"name" : "example local rating like service",
"type" : 7,
"type_pretty" : "local like/dislike rating service"
"type_pretty" : "local like/dislike rating service",
"star_shape" : "circle"
},
"90769255dae5c205c975fc4ce2efff796b8be8a421f786c1737f87f98187ffaf" : {
"name" : "example local rating numerical service",
"type" : 6,
"type_pretty" : "local numerical rating service"
"type_pretty" : "local numerical rating service",
"star_shape" : "fat star",
"min_stars" : 1,
"max_stars" : 5
},
"b474e0cbbab02ca1479c12ad985f1c680ea909a54eb028e3ad06750ea40d4106" : {
"name" : "example local rating inc/dec service",
@ -290,9 +294,14 @@ You won't see all of these, but the service `type` enum is:
`type_pretty` is something you can show users. Hydrus uses the same labels in _manage services_ and so on.
If you want to know the services in a client, hit up [/get\_services](#get_services), which simply gives the above. The same structure has recently been added to [/get\_files/file\_metadata](#get_files_file_metadata) for convenience, since that refers to many different services when it is talking about file locations and ratings and so on.
Rating services now have some extra data:
I expect to hang more information off these in future, particularly star info for ratings.
- like/dislike and numerical services have `star_shape`, which is one of `circle | square | fat star | pentagram star`
- numerical services have `min_stars` (0 or 1) and `max_stars` (1 to 20)
If you are displaying ratings, don't feel crazy obligated to obey the shape! Show a 4/5, select from a dropdown list, do whatever you like!
If you want to know the services in a client, hit up [/get\_services](#get_services), which simply gives the above. The same structure has recently been added to [/get\_files/file\_metadata](#get_files_file_metadata) for convenience, since that refers to many different services when it is talking about file locations and ratings and so on.
Note: If you need to do some quick testing, you should be able to copy the `service_key` of any service by hitting the 'copy service key' button in _review services_.
@ -342,7 +351,8 @@ Arguments:
* 5 - Manage Cookies and Headers
* 6 - Manage Database
* 7 - Edit File Notes
* 8 - Manage File Relationships
* 8 - Edit File Relationships
* 9 - Edit File Ratings
``` title="Example request"
/request_new_permissions?name=my%20import%20script&basic_permissions=[0,1]
@ -351,7 +361,9 @@ Arguments:
Response:
: Some JSON with your access key, which is 64 characters of hex. This will not be valid until the user approves the request in the client ui.
```json title="Example response"
{"access_key" : "73c9ab12751dcf3368f028d3abbe1d8e2a3a48d0de25e64f3a8f00f3a1424c57"}
{
"access_key" : "73c9ab12751dcf3368f028d3abbe1d8e2a3a48d0de25e64f3a8f00f3a1424c57"
}
```
### **GET `/session_key`** { id="session_key" }
@ -477,7 +489,9 @@ Arguments (in JSON):
: - `path`: (the path you want to import)
```json title="Example request body"
{"path" : "E:\\to_import\\ayanami.jpg"}
{
"path" : "E:\\to_import\\ayanami.jpg"
}
```
Arguments (as bytes):
@ -524,7 +538,9 @@ Arguments (in JSON):
* `reason`: (optional, string, the reason attached to the delete action)
```json title="Example request body"
{"hash" : "78f92ba4a786225ee2a1236efa6b7dc81dd729faf4af99f96f3e20bad6d8b538"}
{
"hash" : "78f92ba4a786225ee2a1236efa6b7dc81dd729faf4af99f96f3e20bad6d8b538"
}
```
Response:
@ -549,7 +565,9 @@ Arguments (in JSON):
* [file domain](#parameters_file_domain) (optional, defaults to 'all my files')
```json title="Example request body"
{"hash" : "78f92ba4a786225ee2a1236efa6b7dc81dd729faf4af99f96f3e20bad6d8b538"}
{
"hash" : "78f92ba4a786225ee2a1236efa6b7dc81dd729faf4af99f96f3e20bad6d8b538"
}
```
Response:
@ -576,7 +594,9 @@ Arguments (in JSON):
* [files](#parameters_files)
```json title="Example request body"
{"hash" : "78f92ba4a786225ee2a1236efa6b7dc81dd729faf4af99f96f3e20bad6d8b538"}
{
"hash" : "78f92ba4a786225ee2a1236efa6b7dc81dd729faf4af99f96f3e20bad6d8b538"
}
```
Response:
@ -601,7 +621,9 @@ Arguments (in JSON):
* [files](#parameters_files)
```json title="Example request body"
{"hash" : "78f92ba4a786225ee2a1236efa6b7dc81dd729faf4af99f96f3e20bad6d8b538"}
{
"hash" : "78f92ba4a786225ee2a1236efa6b7dc81dd729faf4af99f96f3e20bad6d8b538"
}
```
Response:
@ -996,6 +1018,52 @@ Response description:
So, _do_ be careful about how you spam delete unless it is something that doesn't matter or it is something you'll only be touching again via the API anyway.
## Editing File Ratings
### **POST `/edit_ratings/set_rating`** { id="edit_ratings_set_rating" }
_Add or remove ratings associated with a file._
Restricted access:
: YES. Edit Ratings permission needed.
Required Headers:
:
* `Content-Type`: `application/json`
Arguments (in percent-encoded JSON):
:
* [files](#parameters_files)
* `rating_service_key` : (hexadecimal, the rating service you want to edit)
* `rating` : (mixed datatype, the rating value you want to set)
```json title="Example request body"
{
"hash" : "3b820114f658d768550e4e3d4f1dced3ff8db77443472b5ad93700647ad2d3ba",
"rating_service_key" : "282303611ba853659aa60aeaa5b6312d40e05b58822c52c57ae5e320882ba26e",
"rating" : 2
}
```
This is fairly simple, but there are some caveats around the different rating service types and the actual data you are setting here. It is the same as you'll see in [GET /get\_files/file\_metadata](#get_files_file_metadata).
#### Like/Dislike Ratings
Send `true` for 'like', `false` for 'dislike', or `null` for 'unset'.
#### Numerical Ratings
Send an `int` for the number of stars to set, or `null` for 'unset'.
#### Inc/Dec Ratings
Send an `int` for the number to set. 0 is your minimum.
As with [GET /get\_files/file\_metadata](#get_files_file_metadata), check [The Services Object](#services_object) for the min/max stars on a numerical rating service.
Response:
: 200 and no content.
## Editing File Notes
### **POST `/add_notes/set_notes`** { id="add_notes_set_notes" }
@ -1373,6 +1441,11 @@ Response:
"has_human_readable_embedded_metadata" : true,
"has_icc_profile" : true,
"known_urls" : [],
"ratings" : {
"74d52c6238d25f846d579174c11856b1aaccdb04a185cb2c79f0d0e499284f2c" : null,
"90769255dae5c205c975fc4ce2efff796b8be8a421f786c1737f87f98187ffaf" : null,
"b474e0cbbab02ca1479c12ad985f1c680ea909a54eb028e3ad06750ea40d4106" : 0
},
"tags" : {
"6c6f63616c2074616773" : {
"storage_tags" : {},
@ -1441,6 +1514,11 @@ Response:
"https://img2.gelbooru.com//images/80/c8/80c8646b4a49395fb36c805f316c49a9.jpg",
"http://origin-orig.deviantart.net/ed31/f/2019/210/7/8/beachqueen_samus_by_dandonfuga-ddcu1xg.jpg"
],
"ratings" : {
"74d52c6238d25f846d579174c11856b1aaccdb04a185cb2c79f0d0e499284f2c" : true,
"90769255dae5c205c975fc4ce2efff796b8be8a421f786c1737f87f98187ffaf" : 3,
"b474e0cbbab02ca1479c12ad985f1c680ea909a54eb028e3ad06750ea40d4106" : 11
},
"tags" : {
"6c6f63616c2074616773" : {
"storage_tags" : {
@ -1552,9 +1630,19 @@ The `tags` structure is similar to the [/add\_tags/add\_tags](#add_tags_add_tags
While the 'storage_tags' represent the actual tags stored on the database for a file, 'display_tags' reflect how tags appear in the UI, after siblings are collapsed and parents are added. If you want to edit a file's tags, refer to the storage tags. If you want to render to the user, use the display tags. The display tag calculation logic is very complicated; if the storage tags change, do not try to guess the new display tags yourself--just ask the API again.
#### ratings
The `ratings` structure is simple, but it holds different data types. For each service:
- For a like/dislike service, 'no rating' is null. 'like' is true, 'dislike' is false.
- For a numerical service, 'no rating' is null. Otherwise it will be an integer, for the number of stars.
- For an inc/dec service, it is always an integer. The default value is 0 for all files.
Check [The Services Object](#services_object) to see the shape of a rating star, and min/max number of stars in a numerical service.
#### services
The 'tags' and 'file_services' structures use the hexadecimal `service_key` extensively. If you need to look up the respective service name or type, check [The Services Object](#services_object) under the top level `services` key.
The `tags`, `ratings`, and `file_services` structures use the hexadecimal `service_key` extensively. If you need to look up the respective service name or type, check [The Services Object](#services_object) under the top level `services` key.
!!! note
If you look, those file structures actually include the service name and type already, but this bloated data is deprecated and will be deleted in 2024, so please transition over.

41
docs/old_changelog.html
View File

@ -34,6 +34,47 @@
<div class="content">
<h1 id="changelog"><a href="#changelog">changelog</a></h1>
<ul>
<li>
<h2 id="version_534"><a href="#version_534">version 534</a></h2>
<ul>
<li><h3>user submissions</h3></li>
<li>thanks to a user, we now have SAI2 (.sai2) file support!</li>
<li>thanks to a user, the duplicate filter now says if one file has audio. this complements the recent Hydrus Video Deduplicator (https://github.com/appleappleapplenanner/hydrus-video-deduplicator), which can queue videos up in your dupe filter</li>
<li>thanks to a user, we now have some nice svg images in the help->links(?) menu instead of gritty bitmaps</li>
<li>thanks to a user, some help documentation for recent client vs hydrus_client changes got fixed</li>
<li><h3>quality of life/new stuff</h3></li>
<li>the media viewer's top-area 'removed from x' lines for files deleted from a local file service no longer appear--unless that file is currently in the trash. on clients with busy multiple local file services, they were mostly just annoying and spammy. if you need this data, hit up the right-click menu of the file--it is still listed there</li>
<li>the 'loading' media page now draws a background in the same colour as the thumbnail grid, so new searches or refreshes will no longer flash to a default grey colour--it should just be a smooth thumbs gone/thumbs back now</li>
<li>added a new shortcut action, 'copy small bmp of image for quick source lookups', for last week's new bitmap copy action</li>
<li>it turns out PNG and WEBP files can have EXIF data, and our existing scanner works with them, so the EXIF scanner now looks at PNGs and WEBPs too. PNGs appear to be rare, about 1-in-200. I will retroactively scan your existing WEBPs, since they have EXIF more commonly, maybe as high as 1-in-5, and are less common as a filetype anyway so the scan will be less work, but when you update you will get a yes/no dialog asking if you want to do PNGs too. it isn't a big deal and you can always queue it up later if you want</li>
<li><h3>fixes</h3></li>
<li>I banged my head against the notes layout code and actually had great success--a variety of borked note-spilling-over-into-nothing and note-stretching-itself-crazy and note-has-fifty-pixels-of-margin issues are fixed. let me know if you still have crazy notes anywhere</li>
<li>the duplicate filter right-hand hover is now more aggressive about getting out of the way of the notes hover, especially when the notes hover jitter-resizes itself a few extra pixels of height. the notes hover should no longer ever overlap the duplicate filter hover's top buttons--very sorry this took so long</li>
<li>when you drag and drop thumbnails out of the program while using an automatic pattern to rename them (_options->gui_), all the filenames are now unique, adding '(x)' after their name as needed for dedupe. previously, on duplicates, it was basically doing a weird spam-merge</li>
<li>fixed an issue when sanitizing export filenames when the destination directory's file system type cannot be determined</li>
<li>fixed a bug when doing a search in a deleted file domain with 'file import time' as the file sort</li>
<li>fixed a bug when hitting the shortcut for 'open file in media viewer' on a non-local file</li>
<li>fixed a bug when the client wants to figure out view options for a file that has mime 'application/unknown'</li>
<li>I may have improved the 'woah the db caches are unsynced' error handling in the 'what siblings and parents does this have?' check when you right-click tags</li>
<li><h3>weird bitmap pastes</h3></li>
<li>fixed the new 'paste image' button under `system:similar files` for a certain category of unusual clipboard bitmaps, including several that hydrus itself generates, where it turns out the QImage storage system stores extra padding bytes on each line of pixels</li>
<li>fixed the new 'paste image' button when the incoming bitmap has a useless alpha channel (e.g. 100% transparent). this was not being stripped like it is for imported images, and so some similar files data was not lining up</li>
<li>many bitmaps copied from other programs like Firefox remain slightly different to what hydrus generates (even though both are at 100% scale). my best guess here is that there is some differing ICC-profile-like colour adjustment happening somewhere, probably either a global browser setting, the browser obeying a global GPU setting, a simply better application of such image metadata on the browser's side, or maybe a stripping of such data, since it seems a 'copy image' event in Firefox also generates and attaches to your clipboard a temporary png file in your temp folder, so maybe the bitmap that we pull from the clipboard is actually generated during some conversion process amidst all that, and it loses some jpeg colour data. whatever the case here, it changes the pixel hash and subtly alters the perceptual hash in many cases. I'm bumping the default distance on this search predicate up to 8 now, to catch the weirder instances</li>
<li><h3>misc</h3></li>
<li>the 'does the db partition have 500MB free?' check that runs on database boot now occurs after some initial database cleanup, and it will use half the total database size instead, if that is smaller than 500MB, down to 64MB (issue #1373)</li>
<li>added a note to the 'running from source' help that the newer mpv dll seems to work on Qt5 and Windows 7 (issue #1338)</li>
<li>the twitter parsers and gugs are removed from the defaults for new users. a shame, but we'll see what happens in future</li>
<li>more misc linting cleanup</li>
<li><h3>ratings on the client api</h3></li>
<li>the services object now shows `star_shape` and `min_stars` and `max_stars` for like/dislike and numerical rating services</li>
<li>the file metadata object now has a 'ratings' key, which lists `rating_service_key->rating` for all the client's rating services. this thing is simple and uses human-friendly values, but it can hold several different data types, so check the help for details and examples</li>
<li>a new permission, 'edit ratings', is added.</li>
<li>a new command, `/edit_ratings/set_rating`, is added. Guess what it does! (issue #343)</li>
<li>the help is updated for these</li>
<li>the unit tests are updated for these</li>
<li>the client api version is now 48</li>
</ul>
</li>
<li>
<h2 id="version_533"><a href="#version_533">version 533</a></h2>
<ul>

9
docs/running_from_source.md
View File

@ -54,12 +54,15 @@ There are three external libraries. You just have to get them and put them in th
1. mpv
1. If you are on Windows 8.1 or older, get [this](https://sourceforge.net/projects/mpv-player-windows/files/libmpv/mpv-dev-x86_64-20210228-git-d1be8bb.7z).
2. If you are on Windows 10 or newer, try [this](https://sourceforge.net/projects/mpv-player-windows/files/libmpv/mpv-dev-x86_64-20220501-git-9ffaa6b.7z).
3. If you want the latest, try [this](https://sourceforge.net/projects/mpv-player-windows/files/libmpv/mpv-dev-x86_64-20230212-git-a40958c.7z), but you have to rename the dll to `mpv-2.dll`.
1. If you are on Windows 8.1 or older, [this](https://sourceforge.net/projects/mpv-player-windows/files/libmpv/mpv-dev-x86_64-20210228-git-d1be8bb.7z) is known safe.
2. If you are on Windows 10 or newer and want the simple answer, try [this](https://sourceforge.net/projects/mpv-player-windows/files/libmpv/mpv-dev-x86_64-20220501-git-9ffaa6b.7z).
3. If you want something newer, then go for [this](https://sourceforge.net/projects/mpv-player-windows/files/libmpv/mpv-dev-x86_64-20230212-git-a40958c.7z), but you have to rename the dll to `mpv-2.dll`.
Then open that archive and place the 'mpv-1.dll' or 'mpv-2.dll' into `install_dir`.
??? info "mpv on older Windows"
I have word that that newer mpv, the API version 2.1 that you have to rename to mpv-2.dll, will work on Qt5 and Windows 7. If this applies to you, feel free to have a play around with different versions here. You'll need the newer mpv choice in the setup-venv script too, which, depending on your situation, may not be possible.
2. SQLite3
Go to `install_dir/static/build_files/windows` and copy 'sqlite3.dll' into `install_dir`.

10
hydrus/client/ClientAPI.py
View File

@ -19,8 +19,9 @@ CLIENT_API_PERMISSION_MANAGE_HEADERS = 5
CLIENT_API_PERMISSION_MANAGE_DATABASE = 6
CLIENT_API_PERMISSION_ADD_NOTES = 7
CLIENT_API_PERMISSION_MANAGE_FILE_RELATIONSHIPS = 8
CLIENT_API_PERMISSION_EDIT_RATINGS = 9
ALLOWED_PERMISSIONS = ( CLIENT_API_PERMISSION_ADD_FILES, CLIENT_API_PERMISSION_ADD_TAGS, CLIENT_API_PERMISSION_ADD_URLS, CLIENT_API_PERMISSION_SEARCH_FILES, CLIENT_API_PERMISSION_MANAGE_PAGES, CLIENT_API_PERMISSION_MANAGE_HEADERS, CLIENT_API_PERMISSION_MANAGE_DATABASE, CLIENT_API_PERMISSION_ADD_NOTES, CLIENT_API_PERMISSION_MANAGE_FILE_RELATIONSHIPS )
ALLOWED_PERMISSIONS = ( CLIENT_API_PERMISSION_ADD_FILES, CLIENT_API_PERMISSION_ADD_TAGS, CLIENT_API_PERMISSION_ADD_URLS, CLIENT_API_PERMISSION_SEARCH_FILES, CLIENT_API_PERMISSION_MANAGE_PAGES, CLIENT_API_PERMISSION_MANAGE_HEADERS, CLIENT_API_PERMISSION_MANAGE_DATABASE, CLIENT_API_PERMISSION_ADD_NOTES, CLIENT_API_PERMISSION_MANAGE_FILE_RELATIONSHIPS, CLIENT_API_PERMISSION_EDIT_RATINGS )
basic_permission_to_str_lookup = {}
@ -32,7 +33,8 @@ basic_permission_to_str_lookup[ CLIENT_API_PERMISSION_MANAGE_PAGES ] = 'manage p
basic_permission_to_str_lookup[ CLIENT_API_PERMISSION_MANAGE_HEADERS ] = 'manage cookies and headers'
basic_permission_to_str_lookup[ CLIENT_API_PERMISSION_MANAGE_DATABASE ] = 'manage database'
basic_permission_to_str_lookup[ CLIENT_API_PERMISSION_ADD_NOTES ] = 'edit file notes'
basic_permission_to_str_lookup[ CLIENT_API_PERMISSION_MANAGE_FILE_RELATIONSHIPS ] = 'manage file relationships'
basic_permission_to_str_lookup[ CLIENT_API_PERMISSION_MANAGE_FILE_RELATIONSHIPS ] = 'edit file relationships'
basic_permission_to_str_lookup[ CLIENT_API_PERMISSION_EDIT_RATINGS ] = 'edit file ratings'
SEARCH_RESULTS_CACHE_TIMEOUT = 4 * 3600
@ -407,9 +409,9 @@ class APIPermissions( HydrusSerialisable.SerialisableBaseNamed ):
with self._lock:
l = sorted( ( basic_permission_to_str_lookup[ p ] for p in self._basic_permissions ) )
sorted_perms = sorted( ( basic_permission_to_str_lookup[ p ] for p in self._basic_permissions ) )
return ', '.join( l )
return ', '.join( sorted_perms )

4
hydrus/client/ClientApplicationCommand.py
View File

@ -152,6 +152,7 @@ SIMPLE_ZOOM_DEFAULT = 143
SIMPLE_SHOW_DUPLICATES = 144
SIMPLE_MANAGE_FILE_TIMESTAMPS = 145
SIMPLE_OPEN_FILE_IN_FILE_EXPLORER = 146
SIMPLE_COPY_LITTLE_BMP = 147
simple_enum_to_str_lookup = {
SIMPLE_ARCHIVE_DELETE_FILTER_BACK : 'archive/delete filter: back',
@ -163,7 +164,8 @@ simple_enum_to_str_lookup = {
SIMPLE_CLOSE_MEDIA_VIEWER : 'close media viewer',
SIMPLE_CLOSE_PAGE : 'close page',
SIMPLE_COPY_BMP : 'copy bmp of image',
SIMPLE_COPY_BMP_OR_FILE_IF_NOT_BMPABLE : 'copy bmp of image, or copy file of other files',
SIMPLE_COPY_LITTLE_BMP : 'copy small bmp of image for quick source lookups',
SIMPLE_COPY_BMP_OR_FILE_IF_NOT_BMPABLE : 'copy bmp of image; otherwise copy file',
SIMPLE_COPY_FILE : 'copy file',
SIMPLE_COPY_MD5_HASH : 'copy md5 hash',
SIMPLE_COPY_PATH : 'copy file paths',

14
hydrus/client/ClientController.py
View File

@ -2179,6 +2179,20 @@ class Controller( HydrusController.HydrusController ):
QP.CallAfter( QW.QApplication.exit )
except HydrusExceptions.DBAccessException as e:
trace = traceback.format_exc()
HydrusData.DebugPrint( trace )
text = 'A serious problem occurred while trying to start the program. Full details have been written to the log. The error is:'
text += '\n' * 2
text += str( e )
self.SafeShowCriticalMessage( 'boot error', text )
QP.CallAfter( QW.QApplication.exit, 1 )
except Exception as e:
text = 'A serious error occurred while trying to start the program. The error will be shown next in a window. More information may have been written to client.log.'

10
hydrus/client/ClientOptions.py
View File

@ -1282,6 +1282,11 @@ class ClientOptions( HydrusSerialisable.SerialisableBase ):
with self._lock:
if mime == HC.APPLICATION_UNKNOWN:
return ( CC.MEDIA_VIEWER_ACTION_DO_NOT_SHOW, False, False )
( media_show_action, media_start_paused, media_start_with_embed, preview_show_action, preview_start_paused, preview_start_with_embed, zoom_info ) = self._GetMediaViewOptions( mime )
( possible_show_actions, can_start_paused, can_start_with_embed ) = CC.media_viewer_capabilities[ mime ]
@ -1366,6 +1371,11 @@ class ClientOptions( HydrusSerialisable.SerialisableBase ):
with self._lock:
if mime == HC.APPLICATION_UNKNOWN:
return ( CC.MEDIA_VIEWER_ACTION_DO_NOT_SHOW, False, False )
( media_show_action, media_start_paused, media_start_with_embed, preview_show_action, preview_start_paused, preview_start_with_embed, zoom_info ) = self._GetMediaViewOptions( mime )
( possible_show_actions, can_start_paused, can_start_with_embed ) = CC.media_viewer_capabilities[ mime ]

4
hydrus/client/ClientPaths.py
View File

@ -6,7 +6,9 @@ from hydrus.core import HydrusPaths
def DeletePath( path, always_delete_fully = False ):
if HC.options[ 'delete_to_recycle_bin' ] == True and not always_delete_fully:
delete_to_recycle_bin = HC.options[ 'delete_to_recycle_bin' ]
if delete_to_recycle_bin and not always_delete_fully:
HydrusPaths.RecyclePath( path )

15
hydrus/client/ClientServices.py
View File

@ -803,12 +803,27 @@ class ServiceLocalRatingNumerical( ServiceLocalRatingStars ):
def ConvertStarsToRating( self, stars: int ) -> float:
if stars > self._num_stars:
stars = self._num_stars
if self._allow_zero:
if stars < 0:
stars = 0
rating = stars / self._num_stars
else:
if stars < 1:
stars = 1
rating = ( stars - 1 ) / ( self._num_stars - 1 )

44
hydrus/client/db/ClientDB.py
View File

@ -9474,6 +9474,50 @@ class DB( HydrusDB.HydrusDB ):
if version == 533:
def ask_what_to_do_png_stuff():
message = 'Hey, v534 adds the ability to see EXIF data in PNG and WEBP files. PNGs with EXIF are generally rare, typically less than one in a hundred. It is usually info about the software that made the PNG.'
message += '\n' * 2
message += 'The client will scan all new PNGs for EXIF. Do you want it to also, over the next few days/weeks in the background, scan all your existing PNG files for EXIF data? It does not take a lot of resources, but it will ultimately load every single PNG you have. If you say no, you can always queue the job up yourself under _database->file maintenance_ later.'
from hydrus.client.gui import ClientGUIDialogsQuick
result = ClientGUIDialogsQuick.GetYesNo( None, message, title = 'Scan PNGs?', yes_label = 'do it', no_label = 'do not do it' )
return result == QW.QDialog.Accepted
try:
do_png_stuff = self._controller.CallBlockingToQt( None, ask_what_to_do_png_stuff )
all_local_hash_ids = self.modules_files_storage.GetCurrentHashIdsList( self.modules_services.combined_local_file_service_id )
with self._MakeTemporaryIntegerTable( all_local_hash_ids, 'hash_id' ) as temp_hash_ids_table_name:
mimes_we_want = [ HC.IMAGE_WEBP ]
if do_png_stuff:
mimes_we_want.append( HC.IMAGE_PNG )
hash_ids = self._STS( self._Execute( 'SELECT hash_id FROM {} CROSS JOIN files_info USING ( hash_id ) WHERE mime IN {};'.format( temp_hash_ids_table_name, HydrusData.SplayListForDB( mimes_we_want ) ) ) )
self.modules_files_maintenance_queue.AddJobs( hash_ids, ClientFiles.REGENERATE_FILE_DATA_JOB_FILE_HAS_EXIF )
except Exception as e:
HydrusData.PrintException( e )
message = 'Some exif-scanning failed to schedule! This is not super important, but hydev would be interested in seeing the error that was printed to the log.'
self.pub_initial_message( message )
self._controller.frame_splash_status.SetTitleText( 'updated db to v{}'.format( HydrusData.ToHumanInt( version + 1 ) ) )
self._Execute( 'UPDATE version SET version = ?;', ( version + 1, ) )

2
hydrus/client/db/ClientDBFilesSearch.py
View File

@ -2189,7 +2189,7 @@ class ClientDBFilesQuery( ClientDBModule.ClientDBModule ):
lambdas = [ number_test.GetLambda() for number_test in specific_number_tests ]
megalambda = lambda x: False not in ( l( x ) for l in lambdas )
megalambda = lambda x: False not in ( lamb( x ) for lamb in lambdas )
with self._MakeTemporaryIntegerTable( query_hash_ids, 'hash_id' ) as temp_table_name:

7
hydrus/client/db/ClientDBTagDisplay.py
View File

@ -407,7 +407,9 @@ class ClientDBTagDisplay( ClientDBModule.ClientDBModule ):
existing_tags = { tag for tag in tags if self.modules_tags.TagExists( tag ) }
existing_tag_ids = { self.modules_tags.GetTagId( tag ) for tag in existing_tags }
existing_tag_ids_to_tags = self.modules_tags.GetTagIdsToTags( tags = existing_tags )
existing_tag_ids = set( existing_tag_ids_to_tags.keys() )
tag_ids_to_ideal_tag_ids = self.modules_tag_siblings.GetTagIdsToIdealTagIds( ClientTags.TAG_DISPLAY_ACTUAL, tag_service_id, existing_tag_ids )
@ -427,14 +429,13 @@ class ClientDBTagDisplay( ClientDBModule.ClientDBModule ):
tag_ids_to_tags = self.modules_tags_local_cache.GetTagIdsToTags( tag_ids = all_tag_ids )
for tag_id in existing_tag_ids:
for ( tag_id, tag ) in existing_tag_ids_to_tags.items():
ideal_tag_id = tag_ids_to_ideal_tag_ids[ tag_id ]
sibling_chain_ids = ideal_tag_ids_to_sibling_chain_ids[ ideal_tag_id ]
descendant_tag_ids = ideal_tag_ids_to_descendant_tag_ids[ ideal_tag_id ]
ancestor_tag_ids = ideal_tag_ids_to_ancestor_tag_ids[ ideal_tag_id ]
tag = tag_ids_to_tags[ tag_id ]
ideal_tag = tag_ids_to_tags[ ideal_tag_id ]
sibling_chain_members = { tag_ids_to_tags[ sibling_chain_id ] for sibling_chain_id in sibling_chain_ids }
descendants = { tag_ids_to_tags[ descendant_tag_id ] for descendant_tag_id in descendant_tag_ids }

8
hydrus/client/gui/ClientGUIDragDrop.py
View File

@ -85,6 +85,8 @@ def DoFileExportDragDrop( window, page_key, media, alt_down ):
elif discord_dnd_fix_possible and os.path.exists( temp_dir ):
seen_export_filenames = set()
fallback_filename_terms = ClientExportingFiles.ParseExportPhrase( '{hash}' )
try:
@ -106,13 +108,15 @@ def DoFileExportDragDrop( window, page_key, media, alt_down ):
for ( i, ( m, original_path ) ) in enumerate( media_and_original_paths ):
filename = ClientExportingFiles.GenerateExportFilename( temp_dir, m, filename_terms, i + 1 )
filename = ClientExportingFiles.GenerateExportFilename( temp_dir, m, filename_terms, i + 1, do_not_use_filenames = seen_export_filenames )
if filename == HC.mime_ext_lookup[ m.GetMime() ]:
filename = ClientExportingFiles.GenerateExportFilename( temp_dir, m, fallback_filename_terms, i + 1 )
filename = ClientExportingFiles.GenerateExportFilename( temp_dir, m, fallback_filename_terms, i + 1, do_not_use_filenames = seen_export_filenames )
seen_export_filenames.add( filename )
dnd_path = os.path.join( temp_dir, filename )
if not os.path.exists( dnd_path ):

20
hydrus/client/gui/ClientGUIFunctions.py
View File

@ -95,7 +95,25 @@ def ConvertQtImageToNumPy( qt_image: QG.QImage ):
data_bytes = data_bytearray.asstring( height * width * depth )
numpy_image = numpy.fromstring( data_bytes, dtype = 'uint8' ).reshape( ( height, width, depth ) )
if qt_image.bytesPerLine() == width * depth:
numpy_image = numpy.fromstring( data_bytes, dtype = 'uint8' ).reshape( ( height, width, depth ) )
else:
# ok bro, so in some cases a qt_image stores its lines with a bit of \x00 padding. you have a 990-pixel line that is 2970+2 bytes long
# apparently this is system memory storage limitations blah blah blah. it can also happen when you qt_image.copy(), so I guess it makes for pleasant memory layout little-endian something
# so far I have only encountered simple instances of this, with data up front and zero bytes at the end
# so let's just strip it lad
bytes_per_line = qt_image.bytesPerLine()
desired_bytes_per_line = width * depth
excess_bytes_to_trim = bytes_per_line - desired_bytes_per_line
numpy_padded = numpy.fromstring( data_bytes, dtype = 'uint8' ).reshape( ( height, bytes_per_line ) )
numpy_image = numpy_padded[ :, : -excess_bytes_to_trim ].reshape( ( height, width, depth ) )
return numpy_image

1
hydrus/client/gui/ClientGUIShortcuts.py
View File

@ -288,6 +288,7 @@ SHORTCUTS_MEDIA_ACTIONS = [
CAC.SIMPLE_LAUNCH_THE_ARCHIVE_DELETE_FILTER,
CAC.SIMPLE_COPY_BMP,
CAC.SIMPLE_COPY_BMP_OR_FILE_IF_NOT_BMPABLE,
CAC.SIMPLE_COPY_LITTLE_BMP,
CAC.SIMPLE_COPY_FILE,
CAC.SIMPLE_COPY_PATH,
CAC.SIMPLE_COPY_SHA256_HASH,

31
hydrus/client/gui/canvas/ClientGUICanvas.py
View File

@ -384,7 +384,7 @@ class Canvas( CAC.ApplicationCommandProcessorMixin, QW.QWidget ):
if self._current_media is not None:
if self._current_media.GetMime() in HC.IMAGES:
if self._current_media.IsImage():
HG.client_controller.pub( 'clipboard', 'bmp', ( self._current_media, resolution ) )
@ -897,15 +897,32 @@ class Canvas( CAC.ApplicationCommandProcessorMixin, QW.QWidget ):
self._Archive()
elif action == CAC.SIMPLE_COPY_BMP:
elif action in ( CAC.SIMPLE_COPY_BMP, CAC.SIMPLE_COPY_BMP_OR_FILE_IF_NOT_BMPABLE, CAC.SIMPLE_COPY_LITTLE_BMP ):
self._CopyBMPToClipboard()
if self._current_media is None:
return
elif action == CAC.SIMPLE_COPY_BMP_OR_FILE_IF_NOT_BMPABLE:
copied = False
copied = self._CopyBMPToClipboard()
if self._current_media.IsImage():
( width, height ) = self._current_media.GetResolution()
if action == CAC.SIMPLE_COPY_LITTLE_BMP and ( width > 1024 or height > 1024 ):
( clip_rect, clipped_res ) = HydrusImageHandling.GetThumbnailResolutionAndClipRegion( self._current_media.GetResolution(), ( 1024, 1024 ), HydrusImageHandling.THUMBNAIL_SCALE_TO_FIT, 100 )
copied = self._CopyBMPToClipboard( resolution = clipped_res )
else:
copied = self._CopyBMPToClipboard()
if not copied:
if action == CAC.SIMPLE_COPY_BMP_OR_FILE_IF_NOT_BMPABLE and not copied:
self._CopyFileToClipboard()
@ -1549,7 +1566,7 @@ class CanvasPanel( Canvas ):
ClientGUIMenus.AppendMenuItem( copy_menu, 'file_id ({})'.format( hash_id_str ), 'Copy this file\'s internal file/hash_id.', HG.client_controller.pub, 'clipboard', 'text', hash_id_str )
if self._current_media.GetMime() in HC.IMAGES:
if self._current_media.IsImage():
ClientGUIMenus.AppendMenuItem( copy_menu, 'bitmap', 'Copy this file to your clipboard as a bitmap.', self._CopyBMPToClipboard )

36
hydrus/client/gui/canvas/ClientGUICanvasHoverFrames.py
View File

@ -355,6 +355,8 @@ class RatingNumericalCanvas( ClientGUIRatings.RatingNumerical ):
class CanvasHoverFrame( QW.QFrame ):
hoverResizedOrMoved = QC.Signal()
sendApplicationCommand = QC.Signal( CAC.ApplicationCommand )
def __init__( self, parent: QW.QWidget, my_canvas, canvas_key ):
@ -453,13 +455,20 @@ class CanvasHoverFrame( QW.QFrame ):
self.resize( my_ideal_size )
if my_ideal_position != self.pos():
should_move = my_ideal_position != self.pos()
if should_move:
self.move( my_ideal_position )
self._position_initialised = True
if should_resize or should_move:
self.hoverResizedOrMoved.emit()
def eventFilter( self, watched, event ):
@ -1489,7 +1498,7 @@ class NotePanel( QW.QWidget ):
vbox = QP.VBoxLayout( margin = 0 )
QP.AddToLayout( vbox, self._note_name, CC.FLAGS_EXPAND_PERPENDICULAR )
QP.AddToLayout( vbox, self._note_text, CC.FLAGS_EXPAND_PERPENDICULAR )
QP.AddToLayout( vbox, self._note_text, CC.FLAGS_EXPAND_BOTH_WAYS )
self._note_text.setVisible( self._note_visible )
@ -1573,6 +1582,14 @@ class NotePanel( QW.QWidget ):
return self._note_visible
def sizeHint( self ) -> QC.QSize:
width = self.parentWidget().GetNoteWidth()
height = self.heightForWidth( width )
return QC.QSize( width, height )
class CanvasHoverFrameRightNotes( CanvasHoverFrame ):
def __init__( self, parent, my_canvas, top_right_hover: CanvasHoverFrameTopRight, canvas_key ):
@ -1581,7 +1598,7 @@ class CanvasHoverFrameRightNotes( CanvasHoverFrame ):
self._top_right_hover = top_right_hover
self._vbox = QP.VBoxLayout()
self._vbox = QP.VBoxLayout( spacing = 2, margin = 2 )
self._names_to_note_panels = {}
self.setSizePolicy( QW.QSizePolicy.Fixed, QW.QSizePolicy.Expanding )
@ -1638,7 +1655,7 @@ class CanvasHoverFrameRightNotes( CanvasHoverFrame ):
# the problem here is that sizeHint produces what width the static text wants based on its own word wrap rules
# we want to say 'with this fixed width, how tall are we?'
# VBoxLayout doesn't support heightForWidth, but statictext does, so let's hack it
# ideal solution here is to write a new layout that delivers heightforwidth, but lmao. maybe Qt6 will do it
# ideal solution here is to write a new layout that delivers heightforwidth, but lmao. maybe Qt6 will do it. EDIT: It didn't really work?
spacing = self.layout().spacing()
margin = self.layout().contentsMargins().top()
@ -1667,7 +1684,7 @@ class CanvasHoverFrameRightNotes( CanvasHoverFrame ):
note_panel_names_with_hidden_notes = set()
for ( name, note_panel ) in self._names_to_note_panels.items():
for ( name, note_panel ) in list( self._names_to_note_panels.items() ):
if not note_panel.IsNoteVisible():
@ -1714,6 +1731,13 @@ class CanvasHoverFrameRightNotes( CanvasHoverFrame ):
return CanvasHoverFrame._ShouldBeHidden( self )
def GetNoteWidth( self ):
note_panel_width = self.width() - ( self.frameWidth() + self.layout().contentsMargins().left() ) * 2
return note_panel_width
def ProcessContentUpdates( self, service_keys_to_content_updates ):
if self._current_media is not None:
@ -1919,6 +1943,8 @@ class CanvasHoverFrameRightDuplicates( CanvasHoverFrame ):
HG.client_controller.sub( self, 'SetDuplicatePair', 'canvas_new_duplicate_pair' )
HG.client_controller.sub( self, 'SetIndexString', 'canvas_new_index_string' )
self._right_notes_hover.hoverResizedOrMoved.connect( self._SizeAndPosition )
def _EditBackgroundSwitchIntensity( self ):

312
hydrus/client/gui/pages/ClientGUIResults.py
View File

@ -102,6 +102,9 @@ class MediaPanel( CAC.ApplicationCommandProcessorMixin, ClientMedia.ListeningMed
self._my_shortcut_handler = ClientGUIShortcuts.ShortcutsHandler( self, [ 'media' ] )
self.setWidget( self._InnerWidget( self ) )
self.setWidgetResizable( True )
def __bool__( self ):
@ -974,6 +977,11 @@ class MediaPanel( CAC.ApplicationCommandProcessorMixin, ClientMedia.ListeningMed
media = self._GetFocusSingleton()
if not media.GetLocationsManager().IsLocal():
return
new_options = HG.client_controller.new_options
( media_show_action, media_start_paused, media_start_with_embed ) = new_options.GetMediaShowAction( media.GetMime() )
@ -2044,15 +2052,32 @@ class MediaPanel( CAC.ApplicationCommandProcessorMixin, ClientMedia.ListeningMed
action = command.GetSimpleAction()
if action == CAC.SIMPLE_COPY_BMP:
if action in ( CAC.SIMPLE_COPY_BMP, CAC.SIMPLE_COPY_BMP_OR_FILE_IF_NOT_BMPABLE, CAC.SIMPLE_COPY_LITTLE_BMP ):
self._CopyBMPToClipboard()
if self._focused_media is None:
return
elif action == CAC.SIMPLE_COPY_BMP_OR_FILE_IF_NOT_BMPABLE:
copied = False
copied = self._CopyBMPToClipboard()
if self._focused_media.IsImage():
( width, height ) = self._focused_media.GetResolution()
if action == CAC.SIMPLE_COPY_LITTLE_BMP and ( width > 1024 or height > 1024 ):
( clip_rect, clipped_res ) = HydrusImageHandling.GetThumbnailResolutionAndClipRegion( self._focused_media.GetResolution(), ( 1024, 1024 ), HydrusImageHandling.THUMBNAIL_SCALE_TO_FIT, 100 )
copied = self._CopyBMPToClipboard( resolution = clipped_res )
else:
copied = self._CopyBMPToClipboard()
if not copied:
if action == CAC.SIMPLE_COPY_BMP_OR_FILE_IF_NOT_BMPABLE and not copied:
self._CopyFilesToClipboard()
@ -2437,6 +2462,38 @@ class MediaPanel( CAC.ApplicationCommandProcessorMixin, ClientMedia.ListeningMed
pass
class _InnerWidget( QW.QWidget ):
def __init__( self, parent ):
QW.QWidget.__init__( self, parent )
self._parent = parent
def paintEvent( self, event ):
painter = QG.QPainter( self )
bg_colour = HG.client_controller.new_options.GetColour( CC.COLOUR_THUMBGRID_BACKGROUND )
painter.setBackground( QG.QBrush( bg_colour ) )
painter.eraseRect( painter.viewport() )
background_pixmap = HG.client_controller.bitmap_manager.GetMediaBackgroundPixmap()
if background_pixmap is not None:
my_size = QP.ScrollAreaVisibleRect( self._parent ).size()
pixmap_size = background_pixmap.size()
painter.drawPixmap( my_size.width() - pixmap_size.width(), my_size.height() - pixmap_size.height(), background_pixmap )
class MediaPanelLoading( MediaPanel ):
def __init__( self, parent, page_key, management_controller: ClientGUIManagementController.ManagementController ):
@ -2492,8 +2549,6 @@ class MediaPanelThumbnails( MediaPanel ):
self._num_rows_per_canvas_page = 1
self._num_rows_per_actual_page = 1
MediaPanel.__init__( self, parent, page_key, management_controller, media_results )
self._last_size = QC.QSize( 20, 20 )
self._num_columns = 1
@ -2502,15 +2557,14 @@ class MediaPanelThumbnails( MediaPanel ):
self._thumbnails_being_faded_in = {}
self._hashes_faded = set()
MediaPanel.__init__( self, parent, page_key, management_controller, media_results )
self._last_device_pixel_ratio = self.devicePixelRatio()
( thumbnail_span_width, thumbnail_span_height ) = self._GetThumbnailSpanDimensions()
thumbnail_scroll_rate = float( HG.client_controller.new_options.GetString( 'thumbnail_scroll_rate' ) )
self.setWidget( MediaPanelThumbnails._InnerWidget( self ) )
self.setWidgetResizable( True )
self.verticalScrollBar().setSingleStep( int( round( thumbnail_span_height * thumbnail_scroll_rate ) ) )
self._widget_event_filter = QP.WidgetEventFilter( self.widget() )
@ -3348,125 +3402,6 @@ class MediaPanelThumbnails( MediaPanel ):
self._UpdateScrollBars()
class _InnerWidget( QW.QWidget ):
def __init__( self, parent ):
QW.QWidget.__init__( self, parent )
self._parent = parent
def mousePressEvent( self, event ):
self._parent._drag_init_coordinates = QG.QCursor.pos()
thumb = self._parent._GetThumbnailUnderMouse( event )
right_on_whitespace = event.button() == QC.Qt.RightButton and thumb is None
if not right_on_whitespace:
self._parent._HitMedia( thumb, event.modifiers() & QC.Qt.ControlModifier, event.modifiers() & QC.Qt.ShiftModifier )
# this specifically does not scroll to media, as for clicking (esp. double-clicking attempts), the scroll can be jarring
def paintEvent( self, event ):
if self._parent.devicePixelRatio() != self._parent._last_device_pixel_ratio:
self._parent._last_device_pixel_ratio = self._parent.devicePixelRatio()
self._parent._DirtyAllPages()
self._parent._DeleteAllDirtyPages()
painter = QG.QPainter( self )
( thumbnail_span_width, thumbnail_span_height ) = self._parent._GetThumbnailSpanDimensions()
page_height = self._parent._num_rows_per_canvas_page * thumbnail_span_height
page_indices_to_display = self._parent._CalculateVisiblePageIndices()
earliest_page_index_to_display = min( page_indices_to_display )
last_page_index_to_display = max( page_indices_to_display )
page_indices_to_draw = list( page_indices_to_display )
if earliest_page_index_to_display > 0:
page_indices_to_draw.append( earliest_page_index_to_display - 1 )
page_indices_to_draw.append( last_page_index_to_display + 1 )
page_indices_to_draw.sort()
potential_clean_indices_to_steal = [ page_index for page_index in self._parent._clean_canvas_pages.keys() if page_index not in page_indices_to_draw ]
random.shuffle( potential_clean_indices_to_steal )
y_start = self._parent._GetYStart()
earliest_y = y_start
bg_colour = HG.client_controller.new_options.GetColour( CC.COLOUR_THUMBGRID_BACKGROUND )
painter.setBackground( QG.QBrush( bg_colour ) )
painter.eraseRect( painter.viewport() )
background_pixmap = HG.client_controller.bitmap_manager.GetMediaBackgroundPixmap()
if background_pixmap is not None:
my_size = QP.ScrollAreaVisibleRect( self._parent ).size()
pixmap_size = background_pixmap.size()
painter.drawPixmap( my_size.width() - pixmap_size.width(), my_size.height() - pixmap_size.height(), background_pixmap )
for page_index in page_indices_to_draw:
if page_index not in self._parent._clean_canvas_pages:
if len( self._parent._dirty_canvas_pages ) == 0:
if len( potential_clean_indices_to_steal ) > 0:
index_to_steal = potential_clean_indices_to_steal.pop()
self._parent._DirtyPage( index_to_steal )
else:
self._parent._CreateNewDirtyPage()
canvas_page = self._parent._dirty_canvas_pages.pop()
self._parent._DrawCanvasPage( page_index, canvas_page )
self._parent._clean_canvas_pages[ page_index ] = canvas_page
if page_index in page_indices_to_display:
canvas_page = self._parent._clean_canvas_pages[ page_index ]
page_virtual_y = page_height * page_index
painter.drawImage( 0, page_virtual_y, canvas_page )
def EventResize( self, event ):
self._ReinitialisePageCacheIfNeeded()
@ -4577,6 +4512,125 @@ class MediaPanelThumbnails( MediaPanel ):
class _InnerWidget( QW.QWidget ):
def __init__( self, parent ):
QW.QWidget.__init__( self, parent )
self._parent = parent
def mousePressEvent( self, event ):
self._parent._drag_init_coordinates = QG.QCursor.pos()
thumb = self._parent._GetThumbnailUnderMouse( event )
right_on_whitespace = event.button() == QC.Qt.RightButton and thumb is None
if not right_on_whitespace:
self._parent._HitMedia( thumb, event.modifiers() & QC.Qt.ControlModifier, event.modifiers() & QC.Qt.ShiftModifier )
# this specifically does not scroll to media, as for clicking (esp. double-clicking attempts), the scroll can be jarring
def paintEvent( self, event ):
if self._parent.devicePixelRatio() != self._parent._last_device_pixel_ratio:
self._parent._last_device_pixel_ratio = self._parent.devicePixelRatio()
self._parent._DirtyAllPages()
self._parent._DeleteAllDirtyPages()
painter = QG.QPainter( self )
( thumbnail_span_width, thumbnail_span_height ) = self._parent._GetThumbnailSpanDimensions()
page_height = self._parent._num_rows_per_canvas_page * thumbnail_span_height
page_indices_to_display = self._parent._CalculateVisiblePageIndices()
earliest_page_index_to_display = min( page_indices_to_display )
last_page_index_to_display = max( page_indices_to_display )
page_indices_to_draw = list( page_indices_to_display )
if earliest_page_index_to_display > 0:
page_indices_to_draw.append( earliest_page_index_to_display - 1 )
page_indices_to_draw.append( last_page_index_to_display + 1 )
page_indices_to_draw.sort()
potential_clean_indices_to_steal = [ page_index for page_index in self._parent._clean_canvas_pages.keys() if page_index not in page_indices_to_draw ]
random.shuffle( potential_clean_indices_to_steal )
y_start = self._parent._GetYStart()
earliest_y = y_start
bg_colour = HG.client_controller.new_options.GetColour( CC.COLOUR_THUMBGRID_BACKGROUND )
painter.setBackground( QG.QBrush( bg_colour ) )
painter.eraseRect( painter.viewport() )
background_pixmap = HG.client_controller.bitmap_manager.GetMediaBackgroundPixmap()
if background_pixmap is not None:
my_size = QP.ScrollAreaVisibleRect( self._parent ).size()
pixmap_size = background_pixmap.size()
painter.drawPixmap( my_size.width() - pixmap_size.width(), my_size.height() - pixmap_size.height(), background_pixmap )
for page_index in page_indices_to_draw:
if page_index not in self._parent._clean_canvas_pages:
if len( self._parent._dirty_canvas_pages ) == 0:
if len( potential_clean_indices_to_steal ) > 0:
index_to_steal = potential_clean_indices_to_steal.pop()
self._parent._DirtyPage( index_to_steal )
else:
self._parent._CreateNewDirtyPage()
canvas_page = self._parent._dirty_canvas_pages.pop()
self._parent._DrawCanvasPage( page_index, canvas_page )
self._parent._clean_canvas_pages[ page_index ] = canvas_page
if page_index in page_indices_to_display:
canvas_page = self._parent._clean_canvas_pages[ page_index ]
page_virtual_y = page_height * page_index
painter.drawImage( 0, page_virtual_y, canvas_page )
def AddRemoveMenu( win: MediaPanel, menu, filter_counts, all_specific_file_domains, has_local_and_remote ):
file_filter_all = ClientMediaFileFilter.FileFilter( ClientMediaFileFilter.FILE_FILTER_ALL )

4
hydrus/client/gui/search/ClientGUIPredicatesSingle.py
View File

@ -2137,6 +2137,8 @@ class PanelPredicateSystemSimilarToData( PanelPredicateSystemSingle ):
numpy_image = ClientGUIFunctions.ConvertQtImageToNumPy( qt_image )
numpy_image = HydrusImageHandling.StripOutAnyUselessAlphaChannel( numpy_image )
pixel_hash = HydrusImageHandling.GetImagePixelHashNumPy( numpy_image )
perceptual_hashes = ClientImageHandling.GenerateShapePerceptualHashesNumPy( numpy_image )
@ -2218,7 +2220,7 @@ class PanelPredicateSystemSimilarToData( PanelPredicateSystemSingle ):
pixel_hashes = tuple()
perceptual_hashes = tuple()
max_hamming = 4
max_hamming = 8
return ClientSearch.Predicate( ClientSearch.PREDICATE_TYPE_SYSTEM_SIMILAR_TO_DATA, ( pixel_hashes, perceptual_hashes, max_hamming ) )

8
hydrus/client/gui/services/ClientGUIClientsideServices.py
View File

@ -1400,10 +1400,10 @@ class EditServiceStarRatingsSubPanel( ClientGUICommon.StaticBox ):
self._shape = ClientGUICommon.BetterChoice( self )
self._shape.addItem( 'circle', ClientRatings.CIRCLE )
self._shape.addItem( 'square', ClientRatings.SQUARE )
self._shape.addItem( 'fat star', ClientRatings.FAT_STAR )
self._shape.addItem( 'pentagram star', ClientRatings.PENTAGRAM_STAR )
for shape in [ ClientRatings.CIRCLE, ClientRatings.SQUARE, ClientRatings.FAT_STAR, ClientRatings.PENTAGRAM_STAR ]:
self._shape.addItem( ClientRatings.shape_to_str_lookup_dict[ shape ], shape )
#

29
hydrus/client/media/ClientMedia.py
View File

@ -1834,23 +1834,26 @@ class MediaSingleton( Media ):
elif len( deleted_local_file_services ) > 0:
for local_file_service in deleted_local_file_services:
if CC.TRASH_SERVICE_KEY in current_service_keys or not only_interesting_lines:
timestamp = timestamps_manager.GetDeletedTimestamp( local_file_service.GetServiceKey() )
line = 'removed from {} {}'.format( local_file_service.GetName(), ClientTime.TimestampToPrettyTimeDelta( timestamp ) )
if len( deleted_local_file_services ) == 1:
for local_file_service in deleted_local_file_services:
line = f'{line} ({local_file_deletion_reason})'
timestamp = timestamps_manager.GetDeletedTimestamp( local_file_service.GetServiceKey() )
line = 'removed from {} {}'.format( local_file_service.GetName(), ClientTime.TimestampToPrettyTimeDelta( timestamp ) )
if len( deleted_local_file_services ) == 1:
line = f'{line} ({local_file_deletion_reason})'
lines.append( ( True, line ) )
lines.append( ( True, line ) )
if len( deleted_local_file_services ) > 1:
lines.append( ( False, 'Deletion reason: {}'.format( local_file_deletion_reason ) ) )
if len( deleted_local_file_services ) > 1:
lines.append( ( False, 'Deletion reason: {}'.format( local_file_deletion_reason ) ) )

43
hydrus/client/media/ClientMediaManagers.py
View File

@ -1291,7 +1291,7 @@ class NotesManager( object ):
class RatingsManager( object ):
def __init__( self, service_keys_to_ratings: typing.Dict[ bytes, typing.Union[ None, float ] ] ):
def __init__( self, service_keys_to_ratings: typing.Dict[ bytes, typing.Union[ None, float, int ] ] ):
self._service_keys_to_ratings = service_keys_to_ratings
@ -1322,6 +1322,47 @@ class RatingsManager( object ):
def GetRatingForAPI( self, service_key ) -> typing.Union[ int, bool, None ]:
service = HG.client_controller.services_manager.GetService( service_key )
service_type = service.GetServiceType()
if service_key in self._service_keys_to_ratings:
rating = self._service_keys_to_ratings[ service_key ]
if rating is None:
return None
if service_type == HC.LOCAL_RATING_LIKE:
return rating >= 0.5
elif service_type == HC.LOCAL_RATING_NUMERICAL:
return service.ConvertRatingToStars( rating )
elif service_type == HC.LOCAL_RATING_INCDEC:
return int( rating )
else:
if service_type == HC.LOCAL_RATING_INCDEC:
return 0
else:
return None
def GetStarRatingSlice( self, service_keys ):
return frozenset( { self._service_keys_to_ratings[ service_key ] for service_key in service_keys if service_key in self._service_keys_to_ratings } )

2
hydrus/client/media/ClientMediaResult.py
View File

@ -119,7 +119,7 @@ class MediaResult( object ):
return self._file_info_manager.num_words
def GetRatingsManager( self ):
def GetRatingsManager( self ) -> ClientMediaManagers.RatingsManager:
return self._ratings_manager

7
hydrus/client/metadata/ClientRatings.py
View File

@ -12,6 +12,13 @@ SQUARE = 1
FAT_STAR = 2
PENTAGRAM_STAR = 3
shape_to_str_lookup_dict = {
CIRCLE : 'circle',
SQUARE : 'square',
FAT_STAR : 'fat star',
PENTAGRAM_STAR : 'pentagram star'
}
def GetIncDecStateFromMedia( media, service_key ):
values_seen = { m.GetRatingsManager().GetRating( service_key ) for m in media }

6
hydrus/client/networking/ClientLocalServer.py
View File

@ -60,6 +60,12 @@ class HydrusServiceClientAPI( HydrusClientService ):
add_tags = NoResource()
root.putChild( b'edit_ratings', add_tags )
add_tags.putChild( b'set_rating', ClientLocalServerResources.HydrusResourceClientAPIRestrictedEditRatingsSetRating( self._service, self._client_requests_domain ) )
add_tags = NoResource()
root.putChild( b'add_tags', add_tags )
add_tags.putChild( b'add_tags', ClientLocalServerResources.HydrusResourceClientAPIRestrictedAddTagsAddTags( self._service, self._client_requests_domain ) )

150
hydrus/client/networking/ClientLocalServerResources.py
View File

@ -43,6 +43,7 @@ from hydrus.client.importing import ClientImportFiles
from hydrus.client.importing.options import FileImportOptions
from hydrus.client.media import ClientMedia
from hydrus.client.media import ClientMediaFileFilter
from hydrus.client.metadata import ClientRatings
from hydrus.client.metadata import ClientTags
from hydrus.client.networking import ClientNetworkingContexts
from hydrus.client.networking import ClientNetworkingDomain
@ -62,7 +63,7 @@ LOCAL_BOORU_JSON_BYTE_LIST_PARAMS = set()
# if a variable name isn't defined here, a GET with it won't work
CLIENT_API_INT_PARAMS = { 'file_id', 'file_sort_type', 'potentials_search_type', 'pixel_duplicates', 'max_hamming_distance', 'max_num_pairs' }
CLIENT_API_BYTE_PARAMS = { 'hash', 'destination_page_key', 'page_key', 'service_key', 'Hydrus-Client-API-Access-Key', 'Hydrus-Client-API-Session-Key', 'file_service_key', 'deleted_file_service_key', 'tag_service_key', 'tag_service_key_1', 'tag_service_key_2' }
CLIENT_API_BYTE_PARAMS = { 'hash', 'destination_page_key', 'page_key', 'service_key', 'Hydrus-Client-API-Access-Key', 'Hydrus-Client-API-Session-Key', 'file_service_key', 'deleted_file_service_key', 'tag_service_key', 'tag_service_key_1', 'tag_service_key_2', 'rating_service_key' }
CLIENT_API_STRING_PARAMS = { 'name', 'url', 'domain', 'search', 'service_name', 'reason', 'tag_display_type', 'source_hash_type', 'desired_hash_type' }
CLIENT_API_JSON_PARAMS = { 'basic_permissions', 'tags', 'tags_1', 'tags_2', 'file_ids', 'download', 'only_return_identifiers', 'only_return_basic_information', 'create_new_file_ids', 'detailed_url_information', 'hide_service_keys_tags', 'simple', 'file_sort_asc', 'return_hashes', 'return_file_ids', 'include_notes', 'include_services_object', 'notes', 'note_names', 'doublecheck_file_system' }
CLIENT_API_JSON_BYTE_LIST_PARAMS = { 'file_service_keys', 'deleted_file_service_keys', 'hashes' }
@ -186,18 +187,36 @@ def GetServicesDict():
services = HG.client_controller.services_manager.GetServices( service_types )
service_dict = {}
services_dict = {}
for service in services:
service_dict[ service.GetServiceKey().hex() ] = {
service_dict = {
'name' : service.GetName(),
'type' : service.GetServiceType(),
'type_pretty' : HC.service_string_lookup[ service.GetServiceType() ]
}
if service.GetServiceType() in HC.STAR_RATINGS_SERVICES:
shape_label = ClientRatings.shape_to_str_lookup_dict[ service.GetShape() ]
service_dict[ 'star_shape' ] = shape_label
if service.GetServiceType() == HC.LOCAL_RATING_NUMERICAL:
allows_zero = service.AllowZero()
num_stars = service.GetNumStars()
service_dict[ 'min_stars' ] = 0 if allows_zero else 1
service_dict[ 'max_stars' ] = num_stars
services_dict[ service.GetServiceKey().hex() ] = service_dict
return service_dict
return services_dict
def GetServiceKeyFromName( service_name: str ):
@ -1473,6 +1492,7 @@ class HydrusResourceClientAPIRestrictedGetService( HydrusResourceClientAPIRestri
request.client_api_permissions.CheckAtLeastOnePermission(
(
ClientAPI.CLIENT_API_PERMISSION_ADD_FILES,
ClientAPI.CLIENT_API_PERMISSION_EDIT_RATINGS,
ClientAPI.CLIENT_API_PERMISSION_ADD_TAGS,
ClientAPI.CLIENT_API_PERMISSION_ADD_NOTES,
ClientAPI.CLIENT_API_PERMISSION_MANAGE_PAGES,
@ -1560,6 +1580,7 @@ class HydrusResourceClientAPIRestrictedGetServices( HydrusResourceClientAPIRestr
request.client_api_permissions.CheckAtLeastOnePermission(
(
ClientAPI.CLIENT_API_PERMISSION_ADD_FILES,
ClientAPI.CLIENT_API_PERMISSION_EDIT_RATINGS,
ClientAPI.CLIENT_API_PERMISSION_ADD_TAGS,
ClientAPI.CLIENT_API_PERMISSION_ADD_NOTES,
ClientAPI.CLIENT_API_PERMISSION_MANAGE_PAGES,
@ -2186,6 +2207,7 @@ class HydrusResourceClientAPIRestrictedAddTagsCleanTags( HydrusResourceClientAPI
return response_context
class HydrusResourceClientAPIRestrictedAddURLs( HydrusResourceClientAPIRestricted ):
def _CheckAPIPermissions( self, request: HydrusServerRequest.HydrusRequest ):
@ -2193,6 +2215,7 @@ class HydrusResourceClientAPIRestrictedAddURLs( HydrusResourceClientAPIRestricte
request.client_api_permissions.CheckPermission( ClientAPI.CLIENT_API_PERMISSION_ADD_URLS )
class HydrusResourceClientAPIRestrictedAddURLsAssociateURL( HydrusResourceClientAPIRestrictedAddURLs ):
def _threadDoPOSTJob( self, request: HydrusServerRequest.HydrusRequest ):
@ -2278,6 +2301,7 @@ class HydrusResourceClientAPIRestrictedAddURLsAssociateURL( HydrusResourceClient
return response_context
class HydrusResourceClientAPIRestrictedAddURLsGetURLFiles( HydrusResourceClientAPIRestrictedAddURLs ):
def _threadDoGETJob( self, request: HydrusServerRequest.HydrusRequest ):
@ -2451,6 +2475,110 @@ class HydrusResourceClientAPIRestrictedAddURLsImportURL( HydrusResourceClientAPI
return response_context
class HydrusResourceClientAPIRestrictedEditRatings( HydrusResourceClientAPIRestricted ):
def _CheckAPIPermissions( self, request: HydrusServerRequest.HydrusRequest ):
request.client_api_permissions.CheckPermission( ClientAPI.CLIENT_API_PERMISSION_EDIT_RATINGS )
class HydrusResourceClientAPIRestrictedEditRatingsSetRating( HydrusResourceClientAPIRestrictedEditRatings ):
def _threadDoPOSTJob( self, request: HydrusServerRequest.HydrusRequest ):
rating_service_key = request.parsed_request_args.GetValue( 'rating_service_key', bytes )
applicable_hashes = set( ParseHashes( request ) )
if len( applicable_hashes ) == 0:
raise HydrusExceptions.BadRequestException( 'Did not find any hashes to apply the ratings to!' )
if 'rating' not in request.parsed_request_args:
raise HydrusExceptions.BadRequestException( 'Sorry, you need to give a rating to set it to!' )
rating = request.parsed_request_args[ 'rating' ]
rating_service = HG.client_controller.services_manager.GetService( rating_service_key )
rating_service_type = rating_service.GetServiceType()
none_ok = True
if rating_service_type == HC.LOCAL_RATING_LIKE:
expecting_type = bool
elif rating_service_type == HC.LOCAL_RATING_NUMERICAL:
expecting_type = int
elif rating_service_type == HC.LOCAL_RATING_INCDEC:
expecting_type = int
none_ok = False
else:
raise HydrusExceptions.BadRequestException( 'That service is not a rating service!' )
if rating is None:
if not none_ok:
raise HydrusExceptions.BadRequestException( 'Sorry, this service does not allow a null rating!' )
elif not isinstance( rating, expecting_type ):
raise HydrusExceptions.BadRequestException( 'Sorry, this service expects a "{}" rating!'.format( expecting_type.__name__ ) )
rating_for_content_update = rating
if rating_service_type == HC.LOCAL_RATING_LIKE:
if isinstance( rating, bool ):
rating_for_content_update = 1.0 if rating else 0.0
elif rating_service_type == HC.LOCAL_RATING_NUMERICAL:
if isinstance( rating, int ):
rating_for_content_update = rating_service.ConvertStarsToRating( rating )
elif rating_service_type == HC.LOCAL_RATING_INCDEC:
if rating < 0:
rating_for_content_update = 0
content_update = HydrusData.ContentUpdate( HC.CONTENT_TYPE_RATINGS, HC.CONTENT_UPDATE_ADD, ( rating_for_content_update, applicable_hashes ) )
service_keys_to_content_updates = collections.defaultdict( list )
service_keys_to_content_updates[ rating_service_key ].append( content_update )
HG.client_controller.WriteSynchronous( 'content_updates', service_keys_to_content_updates )
response_context = HydrusServerResources.ResponseContext( 200 )
return response_context
class HydrusResourceClientAPIRestrictedGetFiles( HydrusResourceClientAPIRestricted ):
def _CheckAPIPermissions( self, request: HydrusServerRequest.HydrusRequest ):
@ -2762,6 +2890,7 @@ class HydrusResourceClientAPIRestrictedGetFilesFileMetadata( HydrusResourceClien
services_manager = HG.client_controller.services_manager
rating_service_keys = services_manager.GetServiceKeys( HC.RATINGS_SERVICES )
tag_service_keys = services_manager.GetServiceKeys( HC.ALL_TAG_SERVICES )
service_keys_to_types = { service.GetServiceKey() : service.GetServiceType() for service in services_manager.GetServices() }
service_keys_to_names = services_manager.GetServiceKeysToNames()
@ -2905,6 +3034,19 @@ class HydrusResourceClientAPIRestrictedGetFilesFileMetadata( HydrusResourceClien
metadata_row[ 'detailed_known_urls' ] = detailed_known_urls
ratings_manager = media_result.GetRatingsManager()
ratings_dict = {}
for rating_service_key in rating_service_keys:
rating_object = ratings_manager.GetRatingForAPI( rating_service_key )
ratings_dict[ rating_service_key.hex() ] = rating_object
metadata_row[ 'ratings' ] = ratings_dict
tags_manager = media_result.GetTagsManager()
tags_dict = {}

6
hydrus/core/HydrusConstants.py
View File

@ -100,8 +100,8 @@ options = {}
# Misc
NETWORK_VERSION = 20
SOFTWARE_VERSION = 533
CLIENT_API_VERSION = 47
SOFTWARE_VERSION = 534
CLIENT_API_VERSION = 48
SERVER_THUMBNAIL_DIMENSIONS = ( 200, 200 )
@ -764,7 +764,7 @@ MIMES_WITH_THUMBNAILS = set( IMAGES ).union( ANIMATIONS ).union( VIDEO ).union(
FILES_THAT_CAN_HAVE_ICC_PROFILE = { IMAGE_JPEG, IMAGE_PNG, IMAGE_GIF, IMAGE_TIFF }
FILES_THAT_CAN_HAVE_EXIF = { IMAGE_JPEG, IMAGE_TIFF, IMAGE_PNG }
FILES_THAT_CAN_HAVE_EXIF = { IMAGE_JPEG, IMAGE_TIFF, IMAGE_PNG, IMAGE_WEBP }
# images and animations that PIL can handle
FILES_THAT_CAN_HAVE_HUMAN_READABLE_EMBEDDED_METADATA = { IMAGE_JPEG, IMAGE_PNG, IMAGE_BMP, IMAGE_WEBP, IMAGE_TIFF, IMAGE_ICON, IMAGE_GIF, IMAGE_APNG }

21
hydrus/core/HydrusDB.py
View File

@ -135,11 +135,6 @@ class HydrusDB( HydrusDBBase.DBBase ):
def __init__( self, controller, db_dir, db_name ):
if HydrusPaths.GetFreeSpace( db_dir ) < 500 * 1048576:
raise Exception( 'Sorry, it looks like the db partition has less than 500MB, please free up some space.' )
HydrusDBBase.DBBase.__init__( self )
self._controller = controller
@ -221,6 +216,17 @@ class HydrusDB( HydrusDBBase.DBBase ):
self._CloseDBConnection()
total_db_size = self.GetApproxTotalFileSize()
size_check = min( int( total_db_size * 0.5 ), 500 * 1048576 )
size_check = max( size_check, 64 * 1048576 )
if HydrusPaths.GetFreeSpace( db_dir ) < size_check:
raise HydrusExceptions.DBAccessException( 'Sorry, it looks like the database drive partition has less than {} free space. It needs this for database transactions, so please free up some space.'.format( HydrusData.ToHumanBytes( size_check ) ) )
self._InitDB()
( version, ) = self._Execute( 'SELECT version FROM version;' ).fetchone()
@ -691,7 +697,10 @@ class HydrusDB( HydrusDBBase.DBBase ):
path = os.path.join( self._db_dir, filename )
total += os.path.getsize( path )
if os.path.exists( path ):
total += os.path.getsize( path )
return total

2
hydrus/core/HydrusImageHandling.py
View File

@ -529,7 +529,7 @@ def GenerateThumbnailBytesPIL( pil_image: PILImage.Image ) -> bytes:
def GetEXIFDict( pil_image: PILImage.Image ) -> typing.Optional[ dict ]:
if pil_image.format in ( 'JPEG', 'TIFF', 'PNG' ) and hasattr( pil_image, '_getexif' ):
if pil_image.format in ( 'JPEG', 'TIFF', 'PNG', 'WEBP' ) and hasattr( pil_image, '_getexif' ):
try:

13
hydrus/core/HydrusPaths.py
View File

@ -344,7 +344,7 @@ def GetDevice( path ) -> typing.Optional[ str ]:
def GetFileSystemType( path ):
def GetFileSystemType( path ) -> typing.Optional[ str ]:
partition_info = GetPartitionInfo( path )
@ -913,7 +913,16 @@ def SanitizePathForExport( directory_path, directories_and_filename ):
suffix_directories = components[:-1]
force_ntfs = GetFileSystemType( directory_path ).lower() in ( 'ntfs', 'exfat' )
fst = GetFileSystemType( directory_path )
if fst is None:
force_ntfs = False
else:
force_ntfs = fst.lower() in ( 'ntfs', 'exfat' )
suffix_directories = [ SanitizeFilename( suffix_directory, force_ntfs = force_ntfs ) for suffix_directory in suffix_directories ]
filename = SanitizeFilename( filename, force_ntfs = force_ntfs )

315
hydrus/test/TestClientAPI.py
View File

@ -101,12 +101,16 @@ def GetExampleServicesDict():
HG.test_controller.example_like_rating_service_key.hex() : {
'name' : 'example local rating like service',
'type' : 7,
'type_pretty' : 'local like/dislike rating service'
'type_pretty' : 'local like/dislike rating service',
'star_shape' : 'circle'
},
HG.test_controller.example_numerical_rating_service_key.hex() : {
'name' : 'example local rating numerical service',
'type' : 6,
'type_pretty' : 'local numerical rating service'
'type_pretty' : 'local numerical rating service',
'min_stars' : 0,
'max_stars' : 5,
'star_shape' : 'circle'
},
HG.test_controller.example_incdec_rating_service_key.hex() : {
'name' : 'example local rating inc/dec service',
@ -1289,6 +1293,7 @@ class TestClientAPI( unittest.TestCase ):
self._compare_content_updates( service_keys_to_content_updates, expected_service_keys_to_content_updates )
def _test_add_notes( self, connection, set_up_permissions ):
hash = os.urandom( 32 )
@ -1573,6 +1578,280 @@ class TestClientAPI( unittest.TestCase ):
self._compare_content_updates( service_keys_to_content_updates, expected_service_keys_to_content_updates )
def _test_add_ratings( self, connection, set_up_permissions ):
hash = os.urandom( 32 )
hash_hex = hash.hex()
#
api_permissions = set_up_permissions[ 'everything' ]
access_key_hex = api_permissions.GetAccessKey().hex()
headers = { 'Hydrus-Client-API-Access-Key' : access_key_hex, 'Content-Type' : HC.mime_mimetype_string_lookup[ HC.APPLICATION_JSON ] }
# set like like
HG.test_controller.ClearWrites( 'content_updates' )
path = '/edit_ratings/set_rating'
body_dict = { 'hash' : hash_hex, 'rating_service_key' : HG.test_controller.example_like_rating_service_key.hex(), 'rating' : True }
body = json.dumps( body_dict )
connection.request( 'POST', path, body = body, headers = headers )
response = connection.getresponse()
data = response.read()
self.assertEqual( response.status, 200 )
expected_service_keys_to_content_updates = collections.defaultdict( list )
expected_service_keys_to_content_updates[ HG.test_controller.example_like_rating_service_key ] = [ HydrusData.ContentUpdate( HC.CONTENT_TYPE_RATINGS, HC.CONTENT_UPDATE_ADD, ( 1.0, { hash } ) ) ]
[ ( ( service_keys_to_content_updates, ), kwargs ) ] = HG.test_controller.GetWrite( 'content_updates' )
self._compare_content_updates( service_keys_to_content_updates, expected_service_keys_to_content_updates )
# set like dislike
HG.test_controller.ClearWrites( 'content_updates' )
path = '/edit_ratings/set_rating'
body_dict = { 'hash' : hash_hex, 'rating_service_key' : HG.test_controller.example_like_rating_service_key.hex(), 'rating' : False }
body = json.dumps( body_dict )
connection.request( 'POST', path, body = body, headers = headers )
response = connection.getresponse()
data = response.read()
self.assertEqual( response.status, 200 )
expected_service_keys_to_content_updates = collections.defaultdict( list )
expected_service_keys_to_content_updates[ HG.test_controller.example_like_rating_service_key ] = [ HydrusData.ContentUpdate( HC.CONTENT_TYPE_RATINGS, HC.CONTENT_UPDATE_ADD, ( 0.0, { hash } ) ) ]
[ ( ( service_keys_to_content_updates, ), kwargs ) ] = HG.test_controller.GetWrite( 'content_updates' )
self._compare_content_updates( service_keys_to_content_updates, expected_service_keys_to_content_updates )
# set like None
HG.test_controller.ClearWrites( 'content_updates' )
path = '/edit_ratings/set_rating'
body_dict = { 'hash' : hash_hex, 'rating_service_key' : HG.test_controller.example_like_rating_service_key.hex(), 'rating' : None }
body = json.dumps( body_dict )
connection.request( 'POST', path, body = body, headers = headers )
response = connection.getresponse()
data = response.read()
self.assertEqual( response.status, 200 )
expected_service_keys_to_content_updates = collections.defaultdict( list )
expected_service_keys_to_content_updates[ HG.test_controller.example_like_rating_service_key ] = [ HydrusData.ContentUpdate( HC.CONTENT_TYPE_RATINGS, HC.CONTENT_UPDATE_ADD, ( None, { hash } ) ) ]
[ ( ( service_keys_to_content_updates, ), kwargs ) ] = HG.test_controller.GetWrite( 'content_updates' )
self._compare_content_updates( service_keys_to_content_updates, expected_service_keys_to_content_updates )
# set numerical 0
HG.test_controller.ClearWrites( 'content_updates' )
path = '/edit_ratings/set_rating'
body_dict = { 'hash' : hash_hex, 'rating_service_key' : HG.test_controller.example_numerical_rating_service_key.hex(), 'rating' : 0 }
body = json.dumps( body_dict )
connection.request( 'POST', path, body = body, headers = headers )
response = connection.getresponse()
data = response.read()
self.assertEqual( response.status, 200 )
expected_service_keys_to_content_updates = collections.defaultdict( list )
expected_service_keys_to_content_updates[ HG.test_controller.example_numerical_rating_service_key ] = [ HydrusData.ContentUpdate( HC.CONTENT_TYPE_RATINGS, HC.CONTENT_UPDATE_ADD, ( 0.0, { hash } ) ) ]
[ ( ( service_keys_to_content_updates, ), kwargs ) ] = HG.test_controller.GetWrite( 'content_updates' )
self._compare_content_updates( service_keys_to_content_updates, expected_service_keys_to_content_updates )
# set numerical 2 (0.4)
HG.test_controller.ClearWrites( 'content_updates' )
path = '/edit_ratings/set_rating'
body_dict = { 'hash' : hash_hex, 'rating_service_key' : HG.test_controller.example_numerical_rating_service_key.hex(), 'rating' : 2 }
body = json.dumps( body_dict )
connection.request( 'POST', path, body = body, headers = headers )
response = connection.getresponse()
data = response.read()
self.assertEqual( response.status, 200 )
expected_service_keys_to_content_updates = collections.defaultdict( list )
expected_service_keys_to_content_updates[ HG.test_controller.example_numerical_rating_service_key ] = [ HydrusData.ContentUpdate( HC.CONTENT_TYPE_RATINGS, HC.CONTENT_UPDATE_ADD, ( 0.4, { hash } ) ) ]
[ ( ( service_keys_to_content_updates, ), kwargs ) ] = HG.test_controller.GetWrite( 'content_updates' )
self._compare_content_updates( service_keys_to_content_updates, expected_service_keys_to_content_updates )
# set numerical 5 (1.0)
HG.test_controller.ClearWrites( 'content_updates' )
path = '/edit_ratings/set_rating'
body_dict = { 'hash' : hash_hex, 'rating_service_key' : HG.test_controller.example_numerical_rating_service_key.hex(), 'rating' : 5 }
body = json.dumps( body_dict )
connection.request( 'POST', path, body = body, headers = headers )
response = connection.getresponse()
data = response.read()
self.assertEqual( response.status, 200 )
expected_service_keys_to_content_updates = collections.defaultdict( list )
expected_service_keys_to_content_updates[ HG.test_controller.example_numerical_rating_service_key ] = [ HydrusData.ContentUpdate( HC.CONTENT_TYPE_RATINGS, HC.CONTENT_UPDATE_ADD, ( 1.0, { hash } ) ) ]
[ ( ( service_keys_to_content_updates, ), kwargs ) ] = HG.test_controller.GetWrite( 'content_updates' )
self._compare_content_updates( service_keys_to_content_updates, expected_service_keys_to_content_updates )
# set numerical None
HG.test_controller.ClearWrites( 'content_updates' )
path = '/edit_ratings/set_rating'
body_dict = { 'hash' : hash_hex, 'rating_service_key' : HG.test_controller.example_numerical_rating_service_key.hex(), 'rating' : None }
body = json.dumps( body_dict )
connection.request( 'POST', path, body = body, headers = headers )
response = connection.getresponse()
data = response.read()
self.assertEqual( response.status, 200 )
expected_service_keys_to_content_updates = collections.defaultdict( list )
expected_service_keys_to_content_updates[ HG.test_controller.example_numerical_rating_service_key ] = [ HydrusData.ContentUpdate( HC.CONTENT_TYPE_RATINGS, HC.CONTENT_UPDATE_ADD, ( None, { hash } ) ) ]
[ ( ( service_keys_to_content_updates, ), kwargs ) ] = HG.test_controller.GetWrite( 'content_updates' )
self._compare_content_updates( service_keys_to_content_updates, expected_service_keys_to_content_updates )
# set incdec 0
HG.test_controller.ClearWrites( 'content_updates' )
path = '/edit_ratings/set_rating'
body_dict = { 'hash' : hash_hex, 'rating_service_key' : HG.test_controller.example_incdec_rating_service_key.hex(), 'rating' : 0 }
body = json.dumps( body_dict )
connection.request( 'POST', path, body = body, headers = headers )
response = connection.getresponse()
data = response.read()
self.assertEqual( response.status, 200 )
expected_service_keys_to_content_updates = collections.defaultdict( list )
expected_service_keys_to_content_updates[ HG.test_controller.example_incdec_rating_service_key ] = [ HydrusData.ContentUpdate( HC.CONTENT_TYPE_RATINGS, HC.CONTENT_UPDATE_ADD, ( 0, { hash } ) ) ]
[ ( ( service_keys_to_content_updates, ), kwargs ) ] = HG.test_controller.GetWrite( 'content_updates' )
self._compare_content_updates( service_keys_to_content_updates, expected_service_keys_to_content_updates )
# set incdec 5
HG.test_controller.ClearWrites( 'content_updates' )
path = '/edit_ratings/set_rating'
body_dict = { 'hash' : hash_hex, 'rating_service_key' : HG.test_controller.example_incdec_rating_service_key.hex(), 'rating' : 5 }
body = json.dumps( body_dict )
connection.request( 'POST', path, body = body, headers = headers )
response = connection.getresponse()
data = response.read()
self.assertEqual( response.status, 200 )
expected_service_keys_to_content_updates = collections.defaultdict( list )
expected_service_keys_to_content_updates[ HG.test_controller.example_incdec_rating_service_key ] = [ HydrusData.ContentUpdate( HC.CONTENT_TYPE_RATINGS, HC.CONTENT_UPDATE_ADD, ( 5, { hash } ) ) ]
[ ( ( service_keys_to_content_updates, ), kwargs ) ] = HG.test_controller.GetWrite( 'content_updates' )
self._compare_content_updates( service_keys_to_content_updates, expected_service_keys_to_content_updates )
# set incdec -3
HG.test_controller.ClearWrites( 'content_updates' )
path = '/edit_ratings/set_rating'
body_dict = { 'hash' : hash_hex, 'rating_service_key' : HG.test_controller.example_incdec_rating_service_key.hex(), 'rating' : -3 }
body = json.dumps( body_dict )
connection.request( 'POST', path, body = body, headers = headers )
response = connection.getresponse()
data = response.read()
self.assertEqual( response.status, 200 )
expected_service_keys_to_content_updates = collections.defaultdict( list )
expected_service_keys_to_content_updates[ HG.test_controller.example_incdec_rating_service_key ] = [ HydrusData.ContentUpdate( HC.CONTENT_TYPE_RATINGS, HC.CONTENT_UPDATE_ADD, ( 0, { hash } ) ) ]
[ ( ( service_keys_to_content_updates, ), kwargs ) ] = HG.test_controller.GetWrite( 'content_updates' )
self._compare_content_updates( service_keys_to_content_updates, expected_service_keys_to_content_updates )
def _test_add_tags( self, connection, set_up_permissions ):
api_permissions = set_up_permissions[ 'everything' ]
@ -4215,6 +4494,24 @@ class TestClientAPI( unittest.TestCase ):
urls = urls,
service_keys_to_filenames = service_keys_to_filenames
)
ratings_dict = {}
if random.random() > 0.6:
ratings_dict[ HG.test_controller.example_like_rating_service_key ] = 0.0 if random.random() < 0 else 1.0
if random.random() > 0.6:
ratings_dict[ HG.test_controller.example_numerical_rating_service_key ] = random.random()
if random.random() > 0.6:
ratings_dict[ HG.test_controller.example_incdec_rating_service_key ] = int( random.random() * 16 )
ratings_manager = ClientMediaManagers.RatingsManager( {} )
notes_manager = ClientMediaManagers.NotesManager( { 'note' : 'hello', 'note2' : 'hello2' } )
file_viewing_stats_manager = ClientMediaManagers.FileViewingStatsManager.STATICGenerateEmptyManager( timestamps_manager )
@ -4317,6 +4614,19 @@ class TestClientAPI( unittest.TestCase ):
ratings_manager = media_result.GetRatingsManager()
ratings_dict = {}
rating_service_keys = services_manager.GetServiceKeys( HC.RATINGS_SERVICES )
for rating_service_key in rating_service_keys:
ratings_dict[ rating_service_key.hex() ] = ratings_manager.GetRatingForAPI( rating_service_key )
metadata_row[ 'ratings' ] = ratings_dict
tags_manager = media_result.GetTagsManager()
tags_dict = {}
@ -5183,6 +5493,7 @@ class TestClientAPI( unittest.TestCase ):
self._test_add_files_add_file( connection, set_up_permissions )
self._test_add_files_other_actions( connection, set_up_permissions )
self._test_add_notes( connection, set_up_permissions )
self._test_add_ratings( connection, set_up_permissions )
self._test_add_tags( connection, set_up_permissions )
self._test_add_tags_search_tags( connection, set_up_permissions )
self._test_add_urls( connection, set_up_permissions )

BIN
static/default/gugs/twitter profile lookup (with replies).png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 2.7 KiB

BIN
static/default/gugs/twitter profile lookup.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 2.2 KiB

BIN
static/default/parsers/twitter syndication api profile parser.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 7.2 KiB

BIN
static/default/parsers/twitter syndication api timeline-profile parser.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 3.3 KiB

BIN
static/default/parsers/twitter syndication api tweet parser.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 3.4 KiB