david/nostr-rs-relay
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
368 Commits 4 Branches 53 Tags
4121c872bc508e583e44c3a0b7efef33013e0a98
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Greg Heartsfield 6489e685ab refactor: reformat and remove tabs 2023-01-22 10:06:44 -06:00
Greg Heartsfield 6800c2e39d improvement: add NostrRepo trait, with sqlite implementation 
This is inspired by the work of
v0l (https://github.com/v0l/nostr-rs-relay/).

A new trait abstracts the storage layer with an async API.  Rusqlite
is still used with worker threads, but this allows for Postgresql or
other backends to be used.

There may be bugs, this has not been rigorously tested.
 2023-01-22 09:49:49 -06:00
Greg Heartsfield e996d4c009 improvement: default to having a event creation rate limit (5/sec) 2023-01-20 11:10:43 -06:00
Greg Heartsfield 585fdd3884 fix: use data_dir from config.toml if present 
fixes: https://todo.sr.ht/~gheartsfield/nostr-rs-relay/64
 2023-01-16 17:21:12 -06:00
Greg Heartsfield cf3e67500f build: bump version to 0.7.17 2023-01-15 15:48:39 -06:00
Greg Heartsfield 1d19442cfd improvement: upgrade multiple dependencies 
Updating crates.io index
Updating async-trait v0.1.60 -> v0.1.61
Updating axum v0.6.1 -> v0.6.2
Updating axum-core v0.3.0 -> v0.3.1
Updating clap v4.0.32 -> v4.1.1
Updating clap_derive v4.0.21 -> v4.1.0
Updating clap_lex v0.3.0 -> v0.3.1
Updating cxx v1.0.85 -> v1.0.86
Updating cxx-build v1.0.85 -> v1.0.86
Updating cxxbridge-flags v1.0.85 -> v1.0.86
Updating cxxbridge-macro v1.0.85 -> v1.0.86
Updating io-lifetimes v1.0.3 -> v1.0.4
Updating nom v7.1.2 -> v7.1.3
Updating parking_lot_core v0.9.5 -> v0.9.6
Updating pest v2.5.2 -> v2.5.3
Updating pest_derive v2.5.2 -> v2.5.3
Updating pest_generator v2.5.2 -> v2.5.3
Updating pest_meta v2.5.2 -> v2.5.3
Updating prost v0.11.5 -> v0.11.6
Updating prost-derive v0.11.5 -> v0.11.6
Updating prost-types v0.11.5 -> v0.11.6
Updating regex v1.7.0 -> v1.7.1
Updating schannel v0.1.20 -> v0.1.21
Removing sha1 v0.10.5
Adding sha2 v0.10.6
Updating termcolor v1.1.3 -> v1.2.0
Updating tokio v1.23.1 -> v1.24.1
Updating try-lock v0.2.3 -> v0.2.4
Removing windows-sys v0.36.1
Updating windows_aarch64_gnullvm v0.42.0 -> v0.42.1
Removing windows_aarch64_msvc v0.36.1
Removing windows_aarch64_msvc v0.42.0
Adding windows_aarch64_msvc v0.42.1
Removing windows_i686_gnu v0.36.1
Removing windows_i686_gnu v0.42.0
Adding windows_i686_gnu v0.42.1
Removing windows_i686_msvc v0.36.1
Removing windows_i686_msvc v0.42.0
Adding windows_i686_msvc v0.42.1
Removing windows_x86_64_gnu v0.36.1
Removing windows_x86_64_gnu v0.42.0
Adding windows_x86_64_gnu v0.42.1
Updating windows_x86_64_gnullvm v0.42.0 -> v0.42.1
Removing windows_x86_64_msvc v0.36.1
Removing windows_x86_64_msvc v0.42.0
Adding windows_x86_64_msvc v0.42.1
 2023-01-15 15:46:33 -06:00
Greg Heartsfield 13cc24b5cd improvement: log blacklisted events 2023-01-15 15:42:27 -06:00
Greg Heartsfield f543957b34 improvement: clear out hidden events during schema upgrade 2023-01-15 15:27:41 -06:00
Greg Heartsfield 7021f102e8 improvement: delete replaceable events 2023-01-15 15:13:10 -06:00
Greg Heartsfield fddbf321bc perf: add indexes and force their use (authors) 2023-01-15 10:52:49 -06:00
Greg Heartsfield 3e7f2e21df perf: force authors index to be used if possible 2023-01-15 10:23:46 -06:00
Greg Heartsfield 9d9c6c78d1 improvement: refuse to insert events that would automatically be hidden 2023-01-15 10:01:01 -06:00
Greg Heartsfield 703b2efe6e refactor: replaceable check in event 2023-01-15 09:18:53 -06:00
Greg Heartsfield 0db6487ce3 fix: allow tokio tracing to be enabled 
fixes https://github.com/scsibug/nostr-rs-relay/issues/48
 2023-01-14 09:47:23 -06:00
Greg Heartsfield d06d227ebe improvement: lower REQ logging and note possible truncation 2023-01-11 16:56:40 -06:00
Greg Heartsfield 3519488c4e improvement: lower logging for failed REQ parses 2023-01-10 07:41:49 -06:00
Greg Heartsfield fbd3315110 improvement: log REQ messages at debug level 2023-01-09 22:12:20 -06:00
Greg Heartsfield 3d3d1bde53 refactor: clippy suggestions 2023-01-09 22:12:04 -06:00
Greg Heartsfield ed336111bb improvement: alert before long-running migration 2023-01-09 22:11:25 -06:00
Greg Heartsfield 8aed572989 docs: add link to relay setup 2023-01-09 21:33:59 -06:00
Greg Heartsfield 62e8da689d fix: do not force kind_created_at_index when there are tags 2023-01-06 12:57:48 -06:00
Greg Heartsfield 807d1aa384 improvement: log index names used 2023-01-06 12:50:52 -06:00
Greg Heartsfield 66a55b55b9 perf: new index, manually selected when appropriate 2023-01-06 12:17:30 -06:00
Greg Heartsfield 76c77c3e56 feat: bulk loading script for importing events 2023-01-06 12:16:19 -06:00
Greg Heartsfield 50daab8a6f refactor: make a standalone re-tagging function 2023-01-06 06:57:56 -06:00
Greg Heartsfield ffd4e6f997 build: bump version to 0.7.16 2023-01-04 17:28:05 -06:00
Greg Heartsfield bbd716963e improvement: update multiple dependencies 
Updating anyhow v1.0.67 -> v1.0.68
Updating cxx v1.0.84 -> v1.0.85
Updating cxx-build v1.0.84 -> v1.0.85
Updating cxxbridge-flags v1.0.84 -> v1.0.85
Updating cxxbridge-macro v1.0.84 -> v1.0.85
Updating hermit-abi v0.1.19 -> v0.2.6
Updating libc v0.2.138 -> v0.2.139
Updating nom v7.1.1 -> v7.1.2
Updating num_cpus v1.14.0 -> v1.15.0
Updating once_cell v1.16.0 -> v1.17.0
Updating openssl v0.10.44 -> v0.10.45
Updating openssl-sys v0.9.79 -> v0.9.80
Updating pest v2.5.1 -> v2.5.2
Updating pest_derive v2.5.1 -> v2.5.2
Updating pest_generator v2.5.1 -> v2.5.2
Updating pest_meta v2.5.1 -> v2.5.2
Updating proc-macro2 v1.0.48 -> v1.0.49
Updating prost v0.11.3 -> v0.11.5
Updating prost-derive v0.11.2 -> v0.11.5
Updating prost-types v0.11.2 -> v0.11.5
Updating quote v1.0.22 -> v1.0.23
Updating serde v1.0.151 -> v1.0.152
Updating serde_derive v1.0.151 -> v1.0.152
Updating serde_json v1.0.90 -> v1.0.91
Updating syn v1.0.106 -> v1.0.107
Updating tokio v1.23.0 -> v1.23.1
 2023-01-04 17:26:22 -06:00
Greg Heartsfield ca95e8cf22 docs(NIP-26): reflect NIP-26 being disabled in README 2023-01-04 16:54:52 -06:00
Greg Heartsfield e9d2a2cbd0 perf(NIP-26): temporarily disable NIP-26 delegated events 2023-01-04 16:51:22 -06:00
Greg Heartsfield 39a945b493 perf: separate author/delegated_by queries, minor improvement 2023-01-04 16:51:17 -06:00
Greg Heartsfield 9a84dc19e9 perf: author/kind index added (schema v13) 2023-01-04 16:51:02 -06:00
Greg Heartsfield 20c4bb42eb fix: correct log message 2023-01-03 21:24:46 -06:00
Greg Heartsfield 3dd0f2c9c6 fix: do not run auto_vacuum on read-only connections 2023-01-03 17:32:55 -06:00
Greg Heartsfield b7c8737166 improvement: enable auto_vacuum on database creation 2023-01-03 06:22:43 -06:00
Greg Heartsfield c0b112c094 improvement: enable auto_vacuum on connections 2023-01-03 06:22:04 -06:00
Greg Heartsfield cb283ac316 fix: ensure that replaceable events are handled correctly regardless of order receieved 2023-01-02 17:18:11 -06:00
Greg Heartsfield 2c6ac69bfd docs: remove incorrect comment 2023-01-02 15:41:17 -06:00
Greg Heartsfield d929ae2752 improvement: define websocket send queue (unlimited->1024) 2023-01-02 15:39:28 -06:00
Greg Heartsfield 14fe9f9ee1 improvement: remove pauses for backups, likely not needed w/ WAL compaction 2023-01-02 15:38:30 -06:00
Greg Heartsfield 104ef2b9e1 build: bump version to 0.7.15 2022-12-27 17:04:48 -06:00
Greg Heartsfield c06139ec99 docs: start of database maintenance tips 2022-12-27 17:00:14 -06:00
Greg Heartsfield 19ec89593d improvement: drop queries that are running during a checkpoint 2022-12-27 15:24:10 -06:00
Greg Heartsfield 27902bc5f4 improvement: move reader mutex closer to DB connection acquisition 2022-12-27 10:28:56 -06:00
Greg Heartsfield d2adddaee4 improvement: extend allowed wal_checkpoint timeout to 10 sec 2022-12-27 10:13:14 -06:00
Greg Heartsfield b23b3ce8ec improvement: block new readers when WAL is large 2022-12-27 09:48:07 -06:00
Greg Heartsfield 5f9fe1ce59 improvement: do not send realtime only filters to the DB (limit:0) 2022-12-26 12:20:36 -06:00
Greg Heartsfield 6a8c4ed1b5 build: bump version to 0.7.14 2022-12-26 11:26:48 -06:00
Greg Heartsfield 966c853700 docs: non-docker quick start 2022-12-26 10:34:09 -06:00
Greg Heartsfield 65fd0ed08b feat: increase wal_checkpoint time when WAL is large 2022-12-26 10:03:51 -06:00
Greg Heartsfield 0b51675b38 improvement: change suggestion and default for max sqlite DB readers 2022-12-25 11:17:08 -06:00
Greg Heartsfield 2e22334631 refactor: formatting 2022-12-25 11:06:30 -06:00
Greg Heartsfield cb2ac4bf0f improvement: give threads unique names 2022-12-25 10:47:32 -06:00
Greg Heartsfield 38dc7789dc improvement: cleaner slow query logs 2022-12-25 10:47:32 -06:00
Greg Heartsfield ce0e00ffb3 feat: log reader DB pool stats every minute 2022-12-25 10:47:32 -06:00
Greg Heartsfield 3e4ae4aeec feat: cache prepared statements and trace expanded SQL queries 2022-12-25 10:47:32 -06:00
Greg Heartsfield c6a8807485 improvement: send error on empty-string prefix author/id searches 2022-12-25 10:47:32 -06:00
Greg Heartsfield 8137b6211c refactor: clippy suggestions 2022-12-24 10:29:47 -06:00
Greg Heartsfield 29effaae23 build: remove pre-commit rustfmt check 2022-12-24 10:29:30 -06:00
Greg Heartsfield e5074f2e46 feat(NIP-28): replaceable kind 41 channel metadata events 2022-12-24 10:14:43 -06:00
Greg Heartsfield 1e1ec69175 build: remove unnecessary dockerfile mod script 2022-12-23 06:52:09 -06:00
Greg Heartsfield ae0f7171ed build: remove digest-locked docker base images 2022-12-23 06:30:59 -06:00
Greg Heartsfield 4f1a912f36 feat: log origin header from websocket requests 
fixes: https://todo.sr.ht/~gheartsfield/nostr-rs-relay/33
 2022-12-22 16:55:53 -06:00
Greg Heartsfield 95748647f0 build: bump version to 0.7.13 2022-12-22 16:27:34 -06:00
Greg Heartsfield 25480e837f fix: do not block writers for more than 1 second during checkpoints 2022-12-22 16:10:49 -06:00
Greg Heartsfield b80b54cd9d improvement: reduce logging, especially for database pool size 2022-12-22 15:47:33 -06:00
Greg Heartsfield 8ea732cbe5 feat: perform regular database maintenance (60sec), without blocking main writer thread 2022-12-22 15:16:21 -06:00
Greg Heartsfield 0f68c4e5c2 refactor: formatting 2022-12-22 15:15:45 -06:00
Greg Heartsfield dab2cd5792 wip: future changes to rustfmt 2022-12-22 15:13:54 -06:00
Greg Heartsfield f411aa6fc2 fix: do not re-verify NIP-05 entries where metadata was deleted 2022-12-22 13:01:48 -06:00
Greg Heartsfield d31bbda087 improvement: reduce lifetime of database connections 2022-12-22 13:01:12 -06:00
Greg Heartsfield 5917bc53b2 improvement: run maintenance every 60 seconds instead of by event count 2022-12-22 11:40:17 -06:00
Greg Heartsfield 91177c61a1 improvement: log reason for new event creation from nip05 2022-12-22 10:48:30 -06:00
Greg Heartsfield 53c2a8051c improvement: reduce logging 2022-12-22 10:29:27 -06:00
Greg Heartsfield 168cf513ac feat: perform full checkpoints and truncate WAL every 2k events 2022-12-22 10:11:05 -06:00
Greg Heartsfield ea204761c9 fix: do not show slow queries more than once per sub 2022-12-20 15:41:50 -06:00
Greg Heartsfield c270ae1434 improvement: reduce event count for db writer pauses 2022-12-20 15:25:24 -06:00
Greg Heartsfield 64bd983cb6 perf: every 5000 persisted events, pause for 500ms for backups 
I have observed backups running for a very long time under heavy load,
this introduces some artificial delay to give the online backup enough
time to make progress.
 2022-12-20 15:05:04 -06:00
Greg Heartsfield 1c153bc784 perf: shed DB query load when queue gets large 2022-12-20 13:23:21 -06:00
Greg Heartsfield dc11d9a619 improvement: explicitly rollback transaction on duplicate event 2022-12-20 13:23:04 -06:00
Greg Heartsfield cd1557787b improvement: log write pool 2022-12-20 13:21:57 -06:00
Greg Heartsfield 86bb7aeb9a improvement: function to check pool capacity 2022-12-20 10:07:01 -06:00
Greg Heartsfield ce37fc1a2d build: bump version to 0.7.12 2022-12-19 14:50:42 -06:00
Greg Heartsfield 2cfd384339 perf: drop db handles that are not quickly read 2022-12-19 00:18:39 -06:00
Greg Heartsfield 8c013107f9 perf: increase upper bound for sqlite mmap 2022-12-18 23:19:43 -06:00
Greg Heartsfield 64a4466d30 perf: backing down on max_blocking_threads 2022-12-18 23:14:41 -06:00
Greg Heartsfield 1596c23eb4 perf: increase blocking threads now that contention is reduced 2022-12-18 22:46:32 -06:00
Greg Heartsfield 129badd4e1 perf: reduce per thread mmap allocation for DB 2022-12-18 22:45:32 -06:00
Greg Heartsfield 6f7c080180 improvement: reduce number of writer blocking threads from 4->2 2022-12-18 22:32:31 -06:00
Greg Heartsfield af92561ef6 perf: remove shared cache mode (experiment) 2022-12-18 22:15:50 -06:00
Greg Heartsfield d833a3e40d perf: reduce logging 2022-12-18 22:11:46 -06:00
Greg Heartsfield 462eb46642 build: bump version to 0.7.11 2022-12-18 20:52:01 -06:00
Greg Heartsfield cf144d503d perf: reduce logging for slow queries 2022-12-18 20:47:11 -06:00
Greg Heartsfield fb8375aef2 build: bump version to 0.7.10 2022-12-18 13:46:18 -06:00
Greg Heartsfield 88ac31b549 perf: increase channel size for DB communication 2022-12-18 13:44:28 -06:00
Greg Heartsfield 677b7d39e9 improvement: log slow requests that return zero results 2022-12-18 13:42:31 -06:00
Greg Heartsfield b24d2f9aaa perf: set default blocking threads to lower value 2022-12-18 12:20:57 -06:00
Greg Heartsfield 7a3899d852 build: bump version to 0.7.9 2022-12-18 09:21:07 -06:00
Greg Heartsfield 818108b793 improvement: upgrade multiple dependencies 
Updating anyhow v1.0.66 -> v1.0.67
Updating async-trait v0.1.59 -> v0.1.60
Updating cxx v1.0.83 -> v1.0.84
Updating cxx-build v1.0.83 -> v1.0.84
Updating cxxbridge-flags v1.0.83 -> v1.0.84
Updating cxxbridge-macro v1.0.83 -> v1.0.84
Updating itoa v1.0.4 -> v1.0.5
Updating link-cplusplus v1.0.7 -> v1.0.8
Updating proc-macro2 v1.0.47 -> v1.0.48
Updating quote v1.0.21 -> v1.0.22
Updating rustversion v1.0.9 -> v1.0.11
Updating ryu v1.0.11 -> v1.0.12
Updating scratch v1.0.2 -> v1.0.3
Updating serde v1.0.150 -> v1.0.151
Updating serde_derive v1.0.150 -> v1.0.151
Updating serde_json v1.0.89 -> v1.0.90
Updating syn v1.0.105 -> v1.0.106
Updating thiserror v1.0.37 -> v1.0.38
Updating thiserror-impl v1.0.37 -> v1.0.38
Updating unicode-ident v1.0.5 -> v1.0.6
 2022-12-18 09:16:09 -06:00
Greg Heartsfield d10348f7e1 feat: configurable blocking threads 2022-12-18 09:14:04 -06:00
Greg Heartsfield 8598e443d8 wip: add configuration for future feature (client concurrent db limits) 2022-12-17 23:19:48 -06:00