146 Commits

Author SHA1 Message Date
Dylan Knutson
c74cbfe4e0 record user fav scans 2025-06-27 00:15:35 +00:00
Dylan Knutson
c527a05705 fix visual similarity query, visually similar on post show page 2025-06-20 05:50:42 +00:00
Dylan Knutson
7d0d94d761 counter cache for user model 2025-06-17 05:56:11 +00:00
Dylan Knutson
305ddd2cb6 visual fingerprinting 2025-03-11 01:06:58 +00:00
Dylan Knutson
a209c64149 create multiple fingerprints wip 2025-03-09 20:33:49 +00:00
Dylan Knutson
99809041a5 more fingerprint 2025-03-05 19:40:33 +00:00
Dylan Knutson
f845d06267 simplify post file thumbnails 2025-03-04 01:58:23 +00:00
Dylan Knutson
0040bc45a2 firt tests forpost file thumbnail spec 2025-03-03 23:28:05 +00:00
Dylan Knutson
70debe9995 [WIP] perceptual hashing 2025-03-03 17:05:58 +00:00
Dylan Knutson
c6a7b9d49a first pass on ip address roles 2025-03-02 08:21:14 +00:00
Dylan Knutson
b452e04af4 better similarity helpers 2025-02-27 22:55:48 +00:00
Dylan Knutson
99fe3d951c create domain factors, similar posts/users sections implemented 2025-02-27 07:05:51 +00:00
Dylan Knutson
78c358d31f source link url suffix matching 2025-02-22 06:00:11 +00:00
Dylan Knutson
00d90312dc tests for source links 2025-02-22 04:48:14 +00:00
Dylan Knutson
a09ddfa32b kaminari types, move views around 2025-02-13 07:35:08 +00:00
Dylan Knutson
51be64abba backfill user search names 2025-02-12 21:03:53 +00:00
Dylan Knutson
9c38bfce13 backfill fa posted_at field 2025-02-12 05:22:51 +00:00
Dylan Knutson
369fa71007 more migrating views, get user search working 2025-02-12 01:59:33 +00:00
Dylan Knutson
dd14f1d521 more migrating views over to new unified schema 2025-02-07 04:55:46 +00:00
Dylan Knutson
0f657d0b59 migrate inkbunny to domain script 2025-02-05 17:13:00 +00:00
Dylan Knutson
ea20ece8e2 migration script 2025-02-05 03:46:16 +00:00
Dylan Knutson
2a9302ced9 fixed select, pluck 2025-02-04 20:17:27 +00:00
Dylan Knutson
b62f7094f4 basic indexes fixed, migration script 2025-02-04 19:41:30 +00:00
Dylan Knutson
50e8c059e6 only enqueue scan post job if new record 2025-02-03 19:41:16 +00:00
Dylan Knutson
e71cb9bdb2 migrate fa posts to json_attributes 2025-02-02 03:43:19 +00:00
Dylan Knutson
61f7faa8e4 migrate out of e621 post state_detail 2025-01-29 20:17:14 +00:00
Dylan Knutson
870c2beb4e e621 user fav jobs fixes 2025-01-29 07:17:19 +00:00
Dylan Knutson
7578b85e4b e621 fav jobs 2025-01-28 23:28:35 +00:00
Dylan Knutson
9746cefa51 tablespaces, structure.sql 2025-01-27 18:12:18 +00:00
Dylan Knutson
1cbc94b43c remove delayed_job, use structure.sql 2025-01-27 17:05:25 +00:00
Dylan Knutson
258be5583d Enhance Inkbunny job processing and database schema
- Added support for pool-specific API searches in `ApiSearchPageProcessor` by introducing `pool_id` as a parameter.
- Created a new `UpdatePoolJob` class to handle deep updates for pools, including enqueuing jobs for posts needing updates.
- Updated `UpdatePostsJob` to manage missing posts and pools more effectively, ensuring proper job enqueuing.
- Modified the `Domain::Inkbunny::Pool` model to include a reference to `deep_update_log_entry` for tracking updates.
- Updated database schema to add `deep_update_log_entry_id` to the `domain_inkbunny_pools` table.
- Added tests for the new job functionality and ensured correct processing of API responses.

These changes improve the maintainability and robustness of the Inkbunny job processing system.
2025-01-05 20:15:17 +00:00
Dylan Knutson
b35d6878dd Enhance Inkbunny job processing and database schema
- Marked tasks as complete in TODO.md for the Inkbunny index scan job and log attachment features.
- Refactored `UpdatePostsJob` to improve method naming and enhance error handling for missing posts.
- Introduced associations between `Post`, `Pool`, and `PoolJoin` models to manage relationships effectively.
- Updated database schema to include `ib_pool_id` in pools and modified the `domain_inkbunny_pool_joins` table structure.
- Added tests to ensure correct association of posts with pools and validate left/right post IDs in pool joins.
2025-01-05 19:32:58 +00:00
Dylan Knutson
6a5815d186 Add GoodJob logging enhancements and custom styles
- Introduced a new `good_job_custom.css` file for custom styling of GoodJob logs.
- Added a new `pixiv.png` icon for domain-specific logging in the `e621` posts helper.
- Enhanced the `GoodJobHelper` module to parse ANSI escape codes for better log formatting.
- Implemented a new `GoodJobExecutionLogLinesCollection` model to store log lines associated with job executions.
- Updated views to display job execution details and logs with improved formatting and styling.
- Refactored `ColorLogger` to support log line accumulation for better log management.

These changes aim to improve the logging experience and visual representation of job execution details in the GoodJob dashboard.
2025-01-03 05:58:22 +00:00
Dylan Knutson
f079fc0f98 add typed where possible 2025-01-01 03:29:53 +00:00
Dylan Knutson
7339c1153c add first typed files 2025-01-01 02:05:25 +00:00
Dylan Knutson
fcd9a7fbab Refactor Inkbunny job processing and enhance post management
- Updated Inkbunny job classes to streamline argument handling by removing `ignore_signature_args :caused_by_entry`.
- Enhanced `ApiSearchPageProcessor` and `UpdatePostsJob` to include `caused_by_entry` for better logging and tracking of job origins.
- Introduced `deep_update_log_entry` and `shallow_update_log_entry` associations in Inkbunny post and user models for improved tracking of updates.
- Added `posted_at` attribute to `IndexedPost` model, ensuring synchronization with the postable's posted date.
- Enhanced views to display user posts in a more organized manner, including handling cases where media files are missing.
- Improved tests for Inkbunny jobs and models to ensure robust coverage of new functionality and maintainability.
2025-01-01 00:20:33 +00:00
Dylan Knutson
fc8b75f014 Refactor Inkbunny job handling and enhance gallery scanning
- Removed the `ignore_signature_args :caused_by_entry` from multiple Inkbunny job classes to streamline argument handling.
- Introduced `scanned_gallery_at` attribute in the Inkbunny user model to manage gallery scan timing.
- Added `UserGalleryJob` to handle gallery scanning for Inkbunny users, ensuring efficient processing of user submissions.
- Implemented `ApiSearchPageProcessor` for processing API search results, improving modularity and reusability.
- Updated `LatestPostsJob` to utilize the new processor for fetching and processing submissions.
- Enhanced tests for Inkbunny jobs, ensuring robust coverage for new functionality and improved job argument handling.
2024-12-31 20:08:33 +00:00
Dylan Knutson
cc3482d62a Add UserAvatarJob for Inkbunny avatar management and refactor user model 2024-12-30 22:54:23 +00:00
Dylan Knutson
2681502c3e Add Inkbunny post management functionality
- Introduced a new model for managing Inkbunny posts, including creation, updating, and retrieval of post data.
- Implemented a job system for handling updates to posts and files, ensuring efficient processing of submissions.
- Enhanced the GlobalStatesController to manage Inkbunny credentials, allowing users to set either username/password or session ID.
- Updated routes to support Inkbunny post viewing and management, including parameterized routes for post IDs.
- Created policies to manage access to post details based on user roles, ensuring only authorized users can view sensitive information.
- Improved views for displaying Inkbunny posts, including enhanced layouts and user interaction elements.
- Added comprehensive tests for the new functionality, ensuring robust coverage for post management and credential handling.
2024-12-30 08:07:27 +00:00
Dylan Knutson
7bc3ed07ee Add Inkbunny credentials management functionality
- Introduced methods for managing Inkbunny cookies in the GlobalStatesController, including `ib_cookies`, `edit_ib_cookies`, and `update_ib_cookies`.
- Added a new policy for managing Inkbunny cookies, restricting access to admin users.
- Created views for displaying and editing Inkbunny credentials, enhancing user interaction.
- Updated routes to include paths for Inkbunny cookies management.
- Enhanced tests for the new functionality in the GlobalStatesController spec, ensuring proper handling of credentials.
2024-12-30 03:15:08 +00:00
Dylan Knutson
32173b50d8 global state model 2024-12-30 01:19:00 +00:00
Dylan Knutson
ca914dbe25 Update .gitignore, Gemfile, and various job classes for improved functionality
- Added '*.notes.md' and '*.export' to .gitignore to prevent unnecessary files from being tracked.
- Refactored job classes in the Domain::Fa module to enhance logging and job enqueuing processes, including:
  - Improved logging messages in Domain::Fa::Job::Base for better clarity.
  - Added support for a new 'enqueue_favs_scan' option in user job processing.
  - Enhanced the FavsJob to utilize active fav post joins and added a 'removed' flag for better management of favorites.
- Updated user and post models to include scopes for active favorites and improved error handling in user creation.
- Enhanced the page parser to support new formats for favorites pages and added tests for these changes.
2024-12-29 01:26:39 +00:00
Dylan Knutson
7a7cc84b94 more policy and auth work 2024-12-27 20:27:16 +00:00
Dylan Knutson
18b9bf531c initial commit for devise and user auth 2024-12-27 19:03:08 +00:00
Dylan Knutson
5c1807711b Add testing utilities and improve test coverage with FactoryBot integration
- Added `shoulda-matchers` for enhanced RSpec testing capabilities.
- Introduced `factory_bot_rails` for easier test data creation.
- Created factories for `HttpLogEntry`, `BlobEntry`, and `Domain::Fa::Post` models.
- Updated `rails_helper.rb` to include FactoryBot methods and configure Shoulda matchers.
- Enhanced `HttpLogEntry` model with a new `response_size` method.
- Refactored `justfile` to include parallel test execution.
- Improved `Gemfile` and `Gemfile.lock` with new testing gems.
2024-12-27 16:59:27 +00:00
Dylan Knutson
23e7eea815 Remove legacy related code 2024-12-25 23:53:29 +00:00
Dylan Knutson
2381df7d10 indexed posts enums 2024-12-25 21:53:47 +00:00
Dylan Knutson
355da6932b indexed posts, more specs 2024-12-25 17:30:03 +00:00
Dylan Knutson
1c86e73df9 fix e621 scan post job 2024-12-24 06:07:23 +00:00
Dylan Knutson
b5cae62f0e fix sqlite exporter 2024-12-21 09:08:55 +00:00