154 Commits

Author SHA1 Message Date
Dylan Knutson
be8e52c792 post favs view, fa and ib post show pages 2025-02-11 20:31:20 +00:00
Dylan Knutson
dd14f1d521 more migrating views over to new unified schema 2025-02-07 04:55:46 +00:00
Dylan Knutson
ea20ece8e2 migration script 2025-02-05 03:46:16 +00:00
Dylan Knutson
b62f7094f4 basic indexes fixed, migration script 2025-02-04 19:41:30 +00:00
Dylan Knutson
e71cb9bdb2 migrate fa posts to json_attributes 2025-02-02 03:43:19 +00:00
Dylan Knutson
172f47b649 good_job cron for periodic tasks 2025-01-27 18:41:05 +00:00
Dylan Knutson
1cbc94b43c remove delayed_job, use structure.sql 2025-01-27 17:05:25 +00:00
Dylan Knutson
b1cdca1513 Enhance routing configuration for admin features
- Added PgHero engine mount to the admin routes for database performance monitoring.
- Can access log entries page from outside VPN connection
2025-01-20 16:25:04 +00:00
Dylan Knutson
304b9bd5d0 Add HTTP gem for request proxying and enhance application layout 2025-01-04 20:39:37 +00:00
Dylan Knutson
02f40215e9 Enhance strict typing and refactor API and job classes
- Updated `ApiController` in the FA domain to enforce strict typing with Sorbet, including the addition of type signatures and improved method parameters.
- Refactored `users_for_name` method to accept a limit parameter for better control over user search results.
- Enhanced error handling in the `UserTimelineTweetsJob` to ensure proper logging and response management.
- Updated `GalleryDlClient` to include strict typing and improved method signatures for better clarity and maintainability.
- Refactored Prometheus metrics configuration to improve naming consistency and clarity.

These changes aim to improve type safety, maintainability, and robustness across the application.
2025-01-04 18:41:29 +00:00
Dylan Knutson
13f078eb34 Add GoodJob metrics integration and update Prometheus configuration
- Introduced a new `GoodJobMetricsWithQueues` class for tracking GoodJob queue counts using Prometheus.
- Updated `application.rb` to include Prometheus exporter dependencies and configuration.
- Added a new `prometheus_exporter.yml` file to manage Prometheus settings across environments.
- Modified `puma.rb` to remove redundant Prometheus instrumentation code.
- Enabled eager loading in the development environment for better performance.
- Updated GoodJob initializer to start the new metrics tracking.

These changes enhance monitoring capabilities for job processing and improve application performance.
2025-01-04 07:23:08 +00:00
Dylan Knutson
72191c1d93 Add Prometheus monitoring integration and enhance application metrics 2025-01-04 05:59:26 +00:00
Dylan Knutson
2a934490ba Update project configuration and enhance OpenTelemetry integration
- Modified `.gitignore` to include and manage `.devcontainer/signoz/data/*` while preserving `.keep` files.
- Updated `.prettierrc` to include the `@prettier/plugin-xml` plugin and configured XML formatting options.
- Added OpenTelemetry SDK and exporter gems to the `Gemfile` for enhanced monitoring capabilities.
- Removed `package-lock.json` as part of the transition to Yarn for dependency management.
- Enhanced `.devcontainer` configuration with new services for SigNoz, including ClickHouse and related configurations.
- Introduced new ClickHouse configuration files for user and cluster settings.
- Updated Nginx and OpenTelemetry collector configurations to support new logging and monitoring features.
- Improved user experience in the `UserSearchBar` component by updating the placeholder text.

These changes aim to improve project maintainability, monitoring capabilities, and user experience.
2025-01-04 00:55:19 +00:00
Dylan Knutson
513267327d Enhance UserMenu with GoodJob integration and update application layout
- Added `goodJobPath` to `UserMenuProps` and integrated a link to the Jobs Queue for admin users.
- Updated the application layout to pass the new `goodJobPath` to the UserMenu component.
- Configured routes to mount the GoodJob engine for admin users, enabling job management features.

These changes improve the user experience by providing quick access to job management for administrators.
2025-01-03 06:06:24 +00:00
Dylan Knutson
6a5815d186 Add GoodJob logging enhancements and custom styles
- Introduced a new `good_job_custom.css` file for custom styling of GoodJob logs.
- Added a new `pixiv.png` icon for domain-specific logging in the `e621` posts helper.
- Enhanced the `GoodJobHelper` module to parse ANSI escape codes for better log formatting.
- Implemented a new `GoodJobExecutionLogLinesCollection` model to store log lines associated with job executions.
- Updated views to display job execution details and logs with improved formatting and styling.
- Refactored `ColorLogger` to support log line accumulation for better log management.

These changes aim to improve the logging experience and visual representation of job execution details in the GoodJob dashboard.
2025-01-03 05:58:22 +00:00
Dylan Knutson
3a14c6946f Add RSpec-Sorbet integration and enhance type safety across the codebase
- Added `rspec-sorbet` gem to the Gemfile for improved type checking in tests.
- Updated various Ruby files to enforce strict typing with Sorbet, enhancing type safety.
- Refactored job classes and models to include type signatures, ensuring better type checking and documentation.
- Modified tests to utilize RSpec-Sorbet features, improving clarity and maintainability.

These changes aim to enhance the overall stability and maintainability of the codebase.
2025-01-01 21:10:54 +00:00
Dylan Knutson
f079fc0f98 add typed where possible 2025-01-01 03:29:53 +00:00
Dylan Knutson
7339c1153c add first typed files 2025-01-01 02:05:25 +00:00
Dylan Knutson
15c11b2b89 Update blob entry handling and enhance staging configuration
- Changed the staging server port from 3000 to 3001 in the Procfile for better port management.
- Introduced a new BlobEntry model to replace BlobEntryP, ensuring a more consistent data structure across the application.
- Updated various controllers and views to utilize the new BlobEntry model, enhancing data retrieval and rendering processes.
- Added a new BlobEntriesController to manage blob entries, including a show action for retrieving content based on SHA256.
- Enhanced the Rakefile to enqueue periodic jobs for updating posts, improving background processing capabilities.
- Updated routes to reflect the new BlobEntry model and ensure proper resource handling.
- Improved tests for blob entry functionality, ensuring robust coverage and reliability in data handling.
2024-12-30 19:35:27 +00:00
Dylan Knutson
2681502c3e Add Inkbunny post management functionality
- Introduced a new model for managing Inkbunny posts, including creation, updating, and retrieval of post data.
- Implemented a job system for handling updates to posts and files, ensuring efficient processing of submissions.
- Enhanced the GlobalStatesController to manage Inkbunny credentials, allowing users to set either username/password or session ID.
- Updated routes to support Inkbunny post viewing and management, including parameterized routes for post IDs.
- Created policies to manage access to post details based on user roles, ensuring only authorized users can view sensitive information.
- Improved views for displaying Inkbunny posts, including enhanced layouts and user interaction elements.
- Added comprehensive tests for the new functionality, ensuring robust coverage for post management and credential handling.
2024-12-30 08:07:27 +00:00
Dylan Knutson
7bc3ed07ee Add Inkbunny credentials management functionality
- Introduced methods for managing Inkbunny cookies in the GlobalStatesController, including `ib_cookies`, `edit_ib_cookies`, and `update_ib_cookies`.
- Added a new policy for managing Inkbunny cookies, restricting access to admin users.
- Created views for displaying and editing Inkbunny credentials, enhancing user interaction.
- Updated routes to include paths for Inkbunny cookies management.
- Enhanced tests for the new functionality in the GlobalStatesController spec, ensuring proper handling of credentials.
2024-12-30 03:15:08 +00:00
Dylan Knutson
9e8f2651db Refactor FA cookie management and enhance testing
- Updated the FA cookie management in the Scraper::FaHttpClientConfig class to validate cookie formats and handle cookies more robustly.
- Removed the fa.yml configuration file and integrated cookie retrieval directly within the class.
- Added comprehensive tests for cookie handling in the new fa_http_client_config_spec.rb file.
- Updated the Gemfile to include parallel_tests in the test and development group.
- Modified the .rspec file to enable color output for better readability in test results.
- Simplified test commands in the justfile for improved usability.
- Adjusted the rails_helper.rb to ensure the test environment is correctly set up.
2024-12-30 02:00:30 +00:00
Dylan Knutson
ec26e425c6 Add FA Cookies management functionality
- Introduced methods for managing FurAffinity cookies in the GlobalStatesController, including `fa_cookies`, `edit_fa_cookies`, and `update_fa_cookies`.
- Added a new policy for managing FA cookies, restricting access to admin users.
- Created views for displaying and editing FA cookies, enhancing user interaction.
- Updated routes to include paths for FA cookies management.
- Added comprehensive tests for the new functionality in the GlobalStatesController spec.
2024-12-30 01:39:21 +00:00
Dylan Knutson
32173b50d8 global state model 2024-12-30 01:19:00 +00:00
Dylan Knutson
3cfa166b4a Enhance Gemfile, update styles, and improve log entry handling
- Added `db-query-matchers` gem for improved query testing capabilities.
- Updated `sanitize` gem version to `~> 6.1` for better security and features.
- Refactored styles in `application.tailwind.css` for better responsiveness.
- Improved `LogEntriesController` to utilize `response_size` for more accurate data handling.
- Added a new `favorites` action in `Domain::Fa::PostsController` for better user experience.
- Enhanced `fa_post_description_sanitized` method in `Domain::Fa::PostsHelper` for improved HTML sanitization.
- Updated views for `Domain::Fa::Posts` to streamline layout and improve user interaction.
- Improved pagination controls for better navigation across post listings.
2024-12-29 20:30:10 +00:00
Dylan Knutson
ca914dbe25 Update .gitignore, Gemfile, and various job classes for improved functionality
- Added '*.notes.md' and '*.export' to .gitignore to prevent unnecessary files from being tracked.
- Refactored job classes in the Domain::Fa module to enhance logging and job enqueuing processes, including:
  - Improved logging messages in Domain::Fa::Job::Base for better clarity.
  - Added support for a new 'enqueue_favs_scan' option in user job processing.
  - Enhanced the FavsJob to utilize active fav post joins and added a 'removed' flag for better management of favorites.
- Updated user and post models to include scopes for active favorites and improved error handling in user creation.
- Enhanced the page parser to support new formats for favorites pages and added tests for these changes.
2024-12-29 01:26:39 +00:00
Dylan Knutson
7a7cc84b94 more policy and auth work 2024-12-27 20:27:16 +00:00
Dylan Knutson
18b9bf531c initial commit for devise and user auth 2024-12-27 19:03:08 +00:00
Dylan Knutson
5c1807711b Add testing utilities and improve test coverage with FactoryBot integration
- Added `shoulda-matchers` for enhanced RSpec testing capabilities.
- Introduced `factory_bot_rails` for easier test data creation.
- Created factories for `HttpLogEntry`, `BlobEntry`, and `Domain::Fa::Post` models.
- Updated `rails_helper.rb` to include FactoryBot methods and configure Shoulda matchers.
- Enhanced `HttpLogEntry` model with a new `response_size` method.
- Refactored `justfile` to include parallel test execution.
- Improved `Gemfile` and `Gemfile.lock` with new testing gems.
2024-12-27 16:59:27 +00:00
Dylan Knutson
23e7eea815 Remove legacy related code 2024-12-25 23:53:29 +00:00
Dylan Knutson
874a92d1b5 add e621 posts controller, index posts filter view 2024-12-25 23:37:38 +00:00
Dylan Knutson
2381df7d10 indexed posts enums 2024-12-25 21:53:47 +00:00
Dylan Knutson
355da6932b indexed posts, more specs 2024-12-25 17:30:03 +00:00
Dylan Knutson
9265b55876 refactor UserSearchBar, js formatting 2024-12-22 19:10:33 +00:00
Dylan Knutson
432c13be42 upgrade to rails 7.2, ui improvements 2024-12-21 19:47:56 +00:00
Dylan Knutson
dba9fc1d4e fix tailwind suggestions, staging live reload 2024-12-19 19:16:42 +00:00
Dylan Knutson
29f671b41f tailwind updates, inkbunny listing stuff 2024-12-19 06:04:37 +00:00
Dylan Knutson
eca9b2e5cb fix specs, add migrator 2024-12-18 22:53:05 +00:00
Dylan Knutson
276a4adc0c format 2024-12-17 23:09:06 +00:00
Dylan Knutson
9de3c1b5e8 remove proxy code 2024-12-17 17:57:17 +00:00
Dylan Knutson
aad2f29e96 before redoing devcontainer files 2024-12-17 06:34:47 -08:00
Dylan Knutson
752b8c3b2c dockerfile 2023-10-08 19:40:54 -07:00
Dylan Knutson
bab5ff1e34 use influxdb 1.8 2023-08-21 16:05:08 -07:00
Dylan Knutson
48cfa13427 add influxdb 2023-08-21 12:05:19 -07:00
Dylan Knutson
fb1cc88b6f skip api token check for FA search_user_names 2023-08-21 11:01:32 -07:00
Dylan Knutson
33d0e02203 add some caching and preloading to user show page 2023-08-21 09:01:54 -07:00
Dylan Knutson
a4247ff8b0 more useful fa user pages, post pages 2023-08-20 19:42:24 -07:00
Dylan Knutson
0b0acd60f2 pass on fa user search 2023-08-18 19:22:39 -07:00
Dylan Knutson
5315534317 fix up metrics reporting 2023-08-18 17:01:33 -07:00
Dylan Knutson
bdfd057cc8 remove id pk from follows, use composite index 2023-05-23 10:36:21 -07:00