110 Commits

Author SHA1 Message Date
Dylan Knutson
7f7728366b Sep 2025 posted_at fixes 2025-09-07 18:27:22 +00:00
Dylan Knutson
a1fab9e645 bmp support, buggy fa user, url decode usernames 2025-08-16 04:44:04 +00:00
Dylan Knutson
ca937eb2bc process mp4 file thumbnailing 2025-08-14 18:16:14 +00:00
Dylan Knutson
ded26741a8 test: update FA job specs and add test fixtures
- Update FurAffinity favs job spec with improved test coverage
- Update user gallery job spec with enhanced testing
- Add new test fixtures for FA favorites and gallery parsing
- Add minimal test fixtures for better test performance
- Update .cursorrules with latest development guidelines
2025-08-10 19:57:04 +00:00
Dylan Knutson
24a59d50f2 telegram bot logs, first pass 2025-08-05 05:05:21 +00:00
Dylan Knutson
f11a5782e1 fuzzysearch handle no sha256 for file 2025-07-25 04:20:40 +00:00
Dylan Knutson
dffdef51cd backup tor archive scraping 2025-07-25 00:25:12 +00:00
Dylan Knutson
58b8321333 Add FA favorites backfill task with date tracking
- Add BackfillFavsAndDatesTask for systematic favorites/dates processing
- Enhance FavsJob with FavsAndDatesStats for date tracking
- Add supporting utilities and parser improvements

This enables robust collection and backfill of user favorites with associated dates.
2025-07-10 17:07:14 +00:00
Dylan Knutson
7f57d4363f compute timezone offset from difference between requested_at and server local time 2025-07-09 21:57:00 +00:00
Dylan Knutson
7a378c0dd5 organize fa fixture files for job testing 2025-07-08 19:49:48 +00:00
Dylan Knutson
0427ac43c7 bootsnap, spring, sofurry type fixes 2025-07-08 01:25:00 +00:00
Dylan Knutson
5a9efa798d more sofurry fixes 2025-07-01 21:02:23 +00:00
Dylan Knutson
66d1e06b4b sofurry user/gallery/post scanning 2025-07-01 19:16:44 +00:00
Dylan Knutson
ca5649a28f initial sofurry impl 2025-06-28 18:46:45 +00:00
Dylan Knutson
308232e01d retry loading gifs as jpg/png on failure 2025-06-26 18:59:31 +00:00
Dylan Knutson
85c4c7e75c FurArchiver fallback for FA posts missing media 2025-06-24 02:19:51 +00:00
Dylan Knutson
7ee3b30180 incremental favs on user pages, user page enqueue job 2025-06-19 22:35:53 +00:00
Dylan Knutson
9d2bde629a detect encoding for plain text files 2025-06-19 16:07:26 +00:00
Dylan Knutson
94533b6e45 FA submission pages no longer have gender field 2025-06-18 15:53:53 +00:00
Dylan Knutson
305ddd2cb6 visual fingerprinting 2025-03-11 01:06:58 +00:00
Dylan Knutson
55f806c5b4 factor out resizing logic 2025-03-09 22:40:37 +00:00
Dylan Knutson
99809041a5 more fingerprint 2025-03-05 19:40:33 +00:00
Dylan Knutson
7453db4ba5 tweaks for march 2025 user page updates 2025-03-04 17:30:27 +00:00
Dylan Knutson
0040bc45a2 firt tests forpost file thumbnail spec 2025-03-03 23:28:05 +00:00
Dylan Knutson
a98d9c49ee user page job improvement for skipping gallery scans 2025-03-02 19:28:47 +00:00
Dylan Knutson
9256d78bf5 improve cache busting based on policy 2025-03-02 07:45:07 +00:00
Dylan Knutson
7048b3002d better behavior for users that are not found 2025-02-28 21:51:29 +00:00
Dylan Knutson
398abf48a7 fixes for plain text bbcode rendering 2025-02-28 21:26:20 +00:00
Dylan Knutson
79a20afcff incremental impl in user page job 2025-02-28 01:20:04 +00:00
Dylan Knutson
3490e28cb4 enqueue from links 2025-02-25 05:47:44 +00:00
Dylan Knutson
cd60166d16 bbcode, doc parsing, render msword documents 2025-02-22 07:49:21 +00:00
Dylan Knutson
6b1956c029 fix how account state is shown, link to last page scan hle 2025-02-19 23:22:24 +00:00
Dylan Knutson
d8eadfd416 specs for user favs scanning 2025-02-19 20:37:22 +00:00
Dylan Knutson
e71cb9bdb2 migrate fa posts to json_attributes 2025-02-02 03:43:19 +00:00
Dylan Knutson
7578b85e4b e621 fav jobs 2025-01-28 23:28:35 +00:00
Dylan Knutson
4d3a8e9fc6 Enhance PostsHelper and View Logic for Improved Post Metadata Display
- Updated `PostsHelper` to enforce strict typing and added new methods for guessing HTTP log entries related to scanned posts and file downloads.
- Refactored the `post_state_string` method to handle unknown states more gracefully.
- Modified the view template to replace the old scanned and file description logic with links to log entries, providing clearer metadata about post actions.
- Removed deprecated tests related to the old description methods and added new tests for the updated functionality.

These changes improve the clarity and usability of post metadata in the application.
2025-01-20 18:00:08 +00:00
Dylan Knutson
d8ffd9bde1 Fix user gallery job pagination logic and add test for empty submission pages
- Updated pagination condition in `user_gallery_job.rb` to break when the number of pages is less than or equal to the current page.
- Added a new test case in `user_gallery_job_spec.rb` to handle scenarios with empty submission pages, ensuring no new records are created.
- Introduced a fixture file `api_search_empty_submissions.json` to simulate API responses with no submissions.
2025-01-20 16:25:26 +00:00
Dylan Knutson
26196534fd favs backfill job 2025-01-13 03:23:24 +00:00
Dylan Knutson
258be5583d Enhance Inkbunny job processing and database schema
- Added support for pool-specific API searches in `ApiSearchPageProcessor` by introducing `pool_id` as a parameter.
- Created a new `UpdatePoolJob` class to handle deep updates for pools, including enqueuing jobs for posts needing updates.
- Updated `UpdatePostsJob` to manage missing posts and pools more effectively, ensuring proper job enqueuing.
- Modified the `Domain::Inkbunny::Pool` model to include a reference to `deep_update_log_entry` for tracking updates.
- Updated database schema to add `deep_update_log_entry_id` to the `domain_inkbunny_pools` table.
- Added tests for the new job functionality and ensured correct processing of API responses.

These changes improve the maintainability and robustness of the Inkbunny job processing system.
2025-01-05 20:15:17 +00:00
Dylan Knutson
025b4ac8e4 Add JobHelper module and enhance Inkbunny job processing
- Introduced a new `JobHelper` module to encapsulate job retrieval and execution logic, enforcing strict typing with Sorbet.
- Updated `UpdatePostsJob` to improve logging and error handling for file processing, including handling files with null MD5 sums.
- Enhanced validation in `Domain::Inkbunny::File` model to conditionally require presence of certain attributes based on file state.
- Added a new fixture for testing scenarios involving submissions with null MD5 sums, improving test coverage and robustness.

These changes aim to improve type safety, maintainability, and error handling in job processing logic.
2025-01-02 17:52:11 +00:00
Dylan Knutson
8817d21e0f Enhance strict typing and improve job initialization in FA and Inkbunny domains
- Refactored job classes in the FA domain to enforce strict typing with Sorbet, including the addition of type signatures in `FavsJob` and `Base` classes.
- Updated initialization methods to accept variable arguments and ensure proper type handling.
- Improved error handling in `ApiSearchPageProcessor` to gracefully manage null values for `last_file_update_datetime`.
- Added a new fixture for testing scenarios where `last_file_update_datetime` is null, enhancing test coverage.

These changes aim to improve type safety, maintainability, and robustness of the job processing logic.
2025-01-02 02:08:38 +00:00
Dylan Knutson
fc8b75f014 Refactor Inkbunny job handling and enhance gallery scanning
- Removed the `ignore_signature_args :caused_by_entry` from multiple Inkbunny job classes to streamline argument handling.
- Introduced `scanned_gallery_at` attribute in the Inkbunny user model to manage gallery scan timing.
- Added `UserGalleryJob` to handle gallery scanning for Inkbunny users, ensuring efficient processing of user submissions.
- Implemented `ApiSearchPageProcessor` for processing API search results, improving modularity and reusability.
- Updated `LatestPostsJob` to utilize the new processor for fetching and processing submissions.
- Enhanced tests for Inkbunny jobs, ensuring robust coverage for new functionality and improved job argument handling.
2024-12-31 20:08:33 +00:00
Dylan Knutson
52498b3cc2 Refactor tests and enhance functionality for Domain::Fa and Domain::E621
- Removed outdated tests for Domain::Fa::PostsController and Domain::Fa::UsersController.
- Added new tests for Domain::Fa::PostsController to verify successful responses for the show action.
- Introduced a new UsersController spec to test the show action for user retrieval.
- Created a factory for Domain::E621::Post to streamline test data creation.
- Added comprehensive tests for BlobFile model, ensuring correct functionality and file operations.
- Implemented tests for HttpLogEntryHeader and LogStoreSstEntry models to validate header scrubbing and version parsing.
- Deleted obsolete test files and directories to clean up the test suite.
2024-12-30 20:11:06 +00:00
Dylan Knutson
15c11b2b89 Update blob entry handling and enhance staging configuration
- Changed the staging server port from 3000 to 3001 in the Procfile for better port management.
- Introduced a new BlobEntry model to replace BlobEntryP, ensuring a more consistent data structure across the application.
- Updated various controllers and views to utilize the new BlobEntry model, enhancing data retrieval and rendering processes.
- Added a new BlobEntriesController to manage blob entries, including a show action for retrieving content based on SHA256.
- Enhanced the Rakefile to enqueue periodic jobs for updating posts, improving background processing capabilities.
- Updated routes to reflect the new BlobEntry model and ensure proper resource handling.
- Improved tests for blob entry functionality, ensuring robust coverage and reliability in data handling.
2024-12-30 19:35:27 +00:00
Dylan Knutson
ca914dbe25 Update .gitignore, Gemfile, and various job classes for improved functionality
- Added '*.notes.md' and '*.export' to .gitignore to prevent unnecessary files from being tracked.
- Refactored job classes in the Domain::Fa module to enhance logging and job enqueuing processes, including:
  - Improved logging messages in Domain::Fa::Job::Base for better clarity.
  - Added support for a new 'enqueue_favs_scan' option in user job processing.
  - Enhanced the FavsJob to utilize active fav post joins and added a 'removed' flag for better management of favorites.
- Updated user and post models to include scopes for active favorites and improved error handling in user creation.
- Enhanced the page parser to support new formats for favorites pages and added tests for these changes.
2024-12-29 01:26:39 +00:00
Dylan Knutson
652463a273 Add domain icons for InkBunny, Newgrounds, and Patreon; update E621 post helper and improve file URL handling 2024-12-27 22:35:31 +00:00
Dylan Knutson
7e02771431 Add pundit-matchers gem and enhance indexed post handling
- Added `pundit-matchers` gem to improve policy testing capabilities.
- Updated `BlobsController` to support a new "tiny" size option for avatars.
- Enhanced `IndexablePostsHelper` with a `show_path` method for different postable types.
- Refactored `IndexedPost` model to include methods for retrieving artist information and external links.
- Modified `Domain::E621::Post` model to initialize `tags_array` as a hash.
- Updated views for indexed posts to support new display formats (gallery and table).
- Improved test coverage with new user factory and updated specs for controller and job behaviors.
2024-12-27 21:56:26 +00:00
Dylan Knutson
18b9bf531c initial commit for devise and user auth 2024-12-27 19:03:08 +00:00
Dylan Knutson
7e19e78073 Refactor user avatar handling and improve user status display
- Updated user avatar creation method to ensure avatar is always created or retrieved.
- Enhanced user status display in views with improved styling and additional information.
- Added missing commas in various places for better code consistency.
- Improved error handling in user avatar job for better logging and state management.
- Updated tests to reflect changes in avatar handling and user model methods.
2024-12-27 18:40:18 +00:00
Dylan Knutson
5c1807711b Add testing utilities and improve test coverage with FactoryBot integration
- Added `shoulda-matchers` for enhanced RSpec testing capabilities.
- Introduced `factory_bot_rails` for easier test data creation.
- Created factories for `HttpLogEntry`, `BlobEntry`, and `Domain::Fa::Post` models.
- Updated `rails_helper.rb` to include FactoryBot methods and configure Shoulda matchers.
- Enhanced `HttpLogEntry` model with a new `response_size` method.
- Refactored `justfile` to include parallel test execution.
- Improved `Gemfile` and `Gemfile.lock` with new testing gems.
2024-12-27 16:59:27 +00:00