changelog
-
+
version 439
+ - tiled image renderer improvements: +
- I believe I fixed the 'non c-contiguous' crash issue with the new tile renderer. I had encountered this while developing, but it was still happening in rare situations--I _think_ in an unlucky edge case where a zoomed tile had the same resolution as the full image rotated by ninety degrees! there is now an additional catch for this situation, as well, to catch any future logical holes. +
- fixed a bug in the new renderer when copying an image to clipboard +
- I greatly mitigated the tiling artifacts with two changes: +
- - zoomed in tiles are now resized with a padding area of up to 4 pixels, with the actual tile cropped afterwards, which allows bilinear and lancsoz interpolation to get accurate neighbour data and have gradient math line up with neighbouring tiles more accurately +
- - on resize and zoom, media canvases now dynamically change tile size to 'neater' float/integer conversion dimensions to reduce sub-pixel panning alignment artifacts (e.g. if your zoom is 300%, the tile is now going to have a dimension that is a multiple of 3) +
- I hacked in a 'rescue offscreen media' calculation after any zoom event. now, if the window is completely out of view after a zoom, it'll snap to the nearest borders, lining against them or overlapping into a buffer zone depending on the zoom. let me know what you think! +
- I fixed a PyQt5 specific object tracking bug, I think the new renderer now works ok for PyQt5! +
- cleaned up some ugly code in the resize section that may have been resulting in incorrect interpolation algorithm choice in some situations +
- fixed a divide by zero issue when zooming out tiny images hugely (e.g. 32x32 at 1%) +
- media windows now try to have at least 1x1 size, just to catch some other weird error situations +
- similarly, tile and native sample sizes will have a minimum of size 1x1, which should fix issues during a delayed startup (issue #872) +
- cleaned up some misc media viewer and tile renderer code +
- . +
- the rest: +
- I started the next round of database optimisation tech, mostly testing out a pipeline upgrade. autocomplete fetching and wildcard file searching for very large queries should be a little faster to cancel now, and in some situations they should be a little faster. they may be slower for very small jobs, but I expect it to be unnoticeable. if you feel autocomplete is suddenly slow and laggy, let me know! +
- I optimised the basic 'ideal sibling normalisation' database query. this is used in a lot of places, so the little saving here should improve a bunch of work +
- I greatly optimised autocomplete sibling population, particularly for searches with a lot of tag results +
- I brushed up the tag import options UI: changed the 'use defaults' checkbox to a dropdown with clear labels for both modes; renamed the 'fetch tags even if' tag import options to 'force page fetch', which is a better description, and added tooltips to describe their ideal use; added tooltips to blacklist and whitelist; and hid the 'load from defaults' button if not set to view specific options +
- added a 'imgur single media file url' File URL Class, which points to direct file links without a referral header, which should fix some situations where these urls were pointed to by other site parsers +
- collections now store the _most recent_ import timestamp of their contents as the aggregate for time imported. previously they had no value, so would sort randomly with each other. collections therefore now sort by time imported reliably with each other, even if there is no 'correct' answer here +
- these new timestamps and service presence generally, and aggregated archive/inbox status, (all of which can update thumbnail display) is now recalculated when files are removed from the collection. so, hitting _right-click->remove->inbox_ will now update collections with a mix of archived and inboxed to remove the inbox icon immediately +
- as the "Retry has no attribute..." network errors have appeared in new forms, I gave the core of the problem another look. we could never really figure this out, but it seemed to be a network version thread safety issue. I think I have ruled this out, and I now believe these may have been occuring during faulty pickling during network session save/load. I fixed the problem here, so with luck this issue will not reappear--if you have had this a lot, let me know how you get on! +
- I broke the requirements.txt into several variants based on platform. we are going to try to pin down good fixed versions of python-mpv and requests/urllib3 for each platform +
- I also updated the 'running from source' help significantly, moving everything to the requirements.txt and making sections for things like FFMPEG and libmpv +
- Also updated the source and contact help around my work style and contact preferences +
- the test.py file now only does the final input() confirmation if there is an interactive stdin to respond +
version 438
- media viewer: diff --git a/help/contact.html b/help/contact.html index dabd702e..e6d8d892 100755 --- a/help/contact.html +++ b/help/contact.html @@ -9,16 +9,18 @@
- homepage
- github (latest build)
- issue tracker
- 8chan.moe /t/ (Hydrus Network General) (endchan bunker (.org)) -
- tumblr (rss) +
- tumblr (rss)
- new downloads
- old downloads
- ImportError: /home/user/hydrus/libX11.so.6: undefined symbol: xcb_poll_for_reply64
- pip3 install beautifulsoup4 chardet html5lib lxml nose numpy opencv-python-headless six Pillow psutil PyYAML requests Send2Trash service_identity twisted +
- pip3 install -r requirements_windows.txt
- pip3 install qtpy PySide2 -
- pip3 install qtpy PyQtChart PyQt5
-
-
python-mpv - to get nice video and audio support!
-If you are on Linux/macOS, you will likely need the mpv library installed to your system, not just mpv, which is often called 'libmpv1'. You can usually get it with apt.
-
- - lz4 - for some memory compression in the client -
- pylzma - for importing rare ZWS swf files -
- cloudscraper - for attempting to solve CloudFlare check pages -
- pysocks - for socks4/socks5 proxy support (although you may want to try "requests[socks]" instead) -
- >PyOpenSSL - to generate a certificate if you want to run the server or the client api -
- mock httmock pyinstaller - if you want to run test.py and make a build yourself -
- PyWin32 pypiwin32 pywin32-ctypes - helpful to ensure you have if you want to make a build in Windows -
- pip3 install beautifulsoup4 chardet html5lib lxml nose numpy opencv-python-headless six Pillow psutil PyOpenSSL PyYAML requests Send2Trash service_identity twisted qtpy PySide2 python-mpv lz4 pylzma cloudscraper pysocks -
-
+
contact and links
I welcome all your bug reports, questions, ideas, and comments. It is always interesting to see how other people are using my software and what they generally think of it. Most of the changes every week are suggested by users.
You can contact me by email, twitter, tumblr, discord, or the 8chan.moe /t/ thread or Endchan board--I do not mind which. Please know that I have difficulty with social media, and while I try to reply to all messages, it sometimes takes me a while to catch up.
-The Github Issue Tracker was turned off for some time, as it did not fit my workflow and I could not keep up, but it is now running again, managed by a team of volunteer users. Please feel free to submit feature requests there if you are comfortable with Github. I am not socially active on Github, and it is mostly just a mirror of my home dev environment, where I work alone.
-I am on the discord on Saturday afternoon, USA time, if you would like to talk live, and briefly on Wednesday after I put the release out. If that is not a good time for you, feel free to leave me a DM and I will get to you when I can. There are also plenty of other hydrus users who idle who would be happy to help with any sort of support question.
+The Github Issue Tracker was turned off for some time, as it did not fit my workflow and I could not keep up, but it is now running again, managed by a team of volunteer users. Please feel free to submit feature requests there if you are comfortable with Github. I am not socially active on Github, please do not ping me there.
+I am on the discord on Saturday afternoon, USA time, if you would like to talk live, and briefly on Wednesday after I put the release out. If that is not a good time for you, please leave me a DM and I will get to you when I can. There are also plenty of other hydrus users who idle who can help with support questions.
I delete all tweets and resolved email conversations after three months. So, if you think you are waiting for a reply, or I said I was going to work on something you care about and seem to have forgotten, please do nudge me.
-Anyway:
+I am always overwhelmed by work and behind on my messages. This is not to say that I do not enjoy just hanging out or talking about possible new features, but forgive me if some work takes longer than expected or if I cannot get to a particular idea quickly. In the same way, if you encounter actual traceback-raising errors or crashes, there is only one guy to fix it, so I prefer to know ASAP so I can prioritise.
+I work by myself because I have acute difficulty working with others. Please do not spontaneously write long design documents or prepare other work for me--I find it more stressful than helpful, every time, and I won't give it the attention it deserves. If you would like to contribute time to hydrus, the user projects like the downloader repository and wiki help guides always have things to do.
+That said:
But that by simply deleting the libX11.so.6 file in the hydrus install directory, he was able to boot. I presume this meant my hydrus build was then relying on his local libX11.so, which happened to have better API compatibility. If you receive a similar error, you might like to try the same sort of thing. Let me know if you discover anything!
-building on windows
+building packages on windows
Installing some packages on windows with pip may need Visual Studio's C++ Build Tools for your version of python. Although these tools are free, it can be a pain to get them through the official (and often huge) downloader installer from Microsoft. Instead, install Chocolatey and use this one simple line:
choco install -y vcbuildtools visualstudio2017buildtools-
Trust me, this will save a ton of headaches! +
Trust me, just do this, it will save a ton of headaches!
what you will need
-You will need basic python experience, python 3.x and a number of python modules. Most of it you can get through pip.
+You will need basic python experience, python 3.x and a number of python modules, all through pip.
If you are on Linux or macOS, or if you are on Windows and have an existing python you do not want to stomp all over with new modules, I recommend you create a virtual environment:
Note, if you are on Linux, it may be easier to use your package manager instead of messing around with venv. A user has written a great summary with all needed packages here.
If you do want to create a new venv environment:
@@ -33,45 +33,36 @@That '. venv/bin/activate' line turns your venv on, and will be needed every time you run the client.pyw/server.py files. You can easily tuck it into a launch script.
On Windows, the path is venv\Scripts\activate, and the whole deal is done much easier in cmd than Powershell. If you get Powershell by default, just type 'cmd' to get an old fashioned command line. In cmd, the launch command is just 'venv\scripts\activate', no leading period.
-After that, you can go nuts with pip. I think this will do for most systems:
+After that, you can use pip to install everything you need from the appropriate requirements.txt in the base install directory. For instance, for Windows, you would go:
-
-
You may want to do all that in smaller batches.
-You will also need Qt5. Either PySide2 (default) or PyQt5 are supported, through qtpy. You can install, again, with pip:
--
-
-or-
+If you prefer to do things manually, inspect the document and install the modules yourself.
+PyQt5 support
+For Qt, either PySide2 (default) or PyQt5 are supported, through qtpy. For PyQt5, go:
Qt 5.15 currently seems to be working well, but 5.14 caused some trouble.
-And optionally, you can add these packages:
--
-
Here is a masterline with everything for general use:
--
-
For Windows, depending on which compiler you are using, pip can have problems building some modules like lz4 and lxml. This page has a lot of prebuilt binaries--I have found it very helpful many times. You may want to update python's sqlite3.dll as well--you can get it here, and just drop it in C:\Python37\DLLs or wherever you have python installed. I have a fair bit of experience with Windows python, so send me a mail if you need help. -
If you don't have ffmpeg in your PATH and you want to import videos, you will need to put a static FFMPEG executable in the install_dir/bin directory. Have a look at how I do it in the extractable compiled releases if you can't figure it out. On Windows, you can copy the exe from one of those releases, or just download the latest static build right from the FFMPEG site. +
FFMPEG
+If you don't have FFMPEG in your PATH and you want to import anything more fun than jpegs, you will need to put a static FFMPEG executable in your PATH or the install_dir/bin directory. If you can't find a static exe on Windows, you can copy the exe from one of my extractable releases. +
mpv support
+MPV is optional and complicated, but it is great, so it is worth the time to figure out!
+As well as the python wrapper, 'python-mpv' as in the requirements.txt, you also need the underlying library. This is not mpv the program, but 'libmpv', often called 'libmpv1'.
+For Windows, the dll builds are here, although getting the right version for the current wrapper can be difficult (you will get errors when you try to load video if it is not correct). Just put it in your hydrus base install directory. You can also just grab the 'mpv-1.dll' I bundle in my release. In my experience, this works with python-mpv 0.5.2. +
If you are on Linux/macOS, you can usually get 'libmpv1' with apt. You might have to adjust your python-mpv version (e.g. "pip3 install python-mpv==0.4.5") to get it to work.
+SQLite
+If you can, update python's SQLite--it'll improve performance.
+On Windows, get the 64-bit sqlite3.dll here, and just drop it in C:\Python37\DLLs or wherever you have python installed. You'll be overwriting the old file, so make a backup if you want to (I have never had trouble updating like this, however).
+I don't know how to do it for Linux or macOS, so if you do, please let me know!
+additional windows info
+This may not matter any more, but in the old days, Windows pip could have problems building modules like lz4 and lxml. This page has a lot of prebuilt binaries--I have found it very helpful many times.
+I have a fair bit of experience with Windows python, so send me a mail if you need help. +
running it
Once you have everything set up, client.pyw and server.py should look for and run off client.db and server.db just like the executables. They will look in the 'db' directory by default, or anywhere you point them with the "-d" parameter, again just like the executables.
-I develop hydrus on and am most experienced with Windows, so the program is more stable and reasonable on that. I do not have as much experience with Linux or macOS, so I would particularly appreciate your Linux/macOS bug reports and any informed suggestions.
+I develop hydrus on and am most experienced with Windows, so the program is more stable and reasonable on that. I do not have as much experience with Linux or macOS, but I still appreciate and will work on your Linux/macOS bug reports.
my code
-Unlike most software people, I am more INFJ than INTP/J. My coding style is unusual and unprofessional, and everything is pretty much hacked together. Please look through the source if you are interested in how things work and ask me if you don't understand something. I'm constantly throwing new code together and then cleaning and overhauling it down the line.
-I work strictly alone, so while I am very interested in detailed bug reports or suggestions for good libraries to use, I am not looking for pull requests. Everything I do is WTFPL, so feel free to fork and play around with things on your end as much as you like.
+My coding style is unusual and unprofessional. Everything is pretty much hacked together. If you are interested in how things work, please do look through the source and ask me if you don't understand something.
+I'm constantly throwing new code together and then cleaning and overhauling it down the line. I work strictly alone, however, so while I am very interested in detailed bug reports or suggestions for good libraries to use, I am not looking for pull requests or suggestions on style. I know a lot of things are a mess. Everything I do is WTFPL, so feel free to fork and play around with things on your end as much as you like.