Skip to content

Commit 471fa8f

Browse files
samwillissamwillis
authored
E2E Test Suite for Query-Driven Sync (#773)
* Implement end-to-end tests for sync feature Implement end-to-end tests for sync feature Implement end-to-end tests for sync feature chore: revert pnpm-lock.yaml to match query-driven-sync chore: update pnpm-lock.yaml with e2e test pg dependencies bump pnpm/action-setup version fix pnpmlockfile format only run normal tests on test command tidy and fix types fixes ci fix tidy checkpoint cleanup between each test add cleanup between tests all e2e tests pass! cleanup live queries fixes for ci add propper mutations to e2e tests fix ci timouts move the badly named accumulated data tests fix ci flake working query e2e tests fix lint fix assertions Fix race condition between test and logical replication stream Handle isNull and isUndefined in expression helpers Ran prettier. fix ci update docs add suport for not to extractSimpleComparisons and get tests to pass try and fix ci race * increase timeout
1 parent 55ea32e commit 471fa8f

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

49 files changed

+6236
-119
lines changed

‎.github/workflows/e2e-tests.yml‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
name: E2E Tests
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
8+
jobs:
9+
e2e-tests:
10+
name: Run E2E Tests
11+
runs-on: ubuntu-latest
12+
timeout-minutes: 10
13+
14+
steps:
15+
- name: Checkout code
16+
uses: actions/checkout@v4
17+
18+
- name: Setup pnpm
19+
uses: pnpm/action-setup@v4
20+
with:
21+
version: 10.19.0
22+
23+
- name: Setup Node.js
24+
uses: actions/setup-node@v4
25+
with:
26+
node-version: "20"
27+
cache: "pnpm"
28+
29+
- name: Install dependencies
30+
run: pnpm install --frozen-lockfile
31+
32+
- name: Start Docker services
33+
run: |
34+
cd packages/db-collection-e2e/docker
35+
docker compose up -d
36+
echo "Waiting for services to be healthy..."
37+
timeout 60 bash -c 'until docker compose ps | grep -q "healthy"; do sleep 2; done'
38+
39+
- name: Build packages
40+
run: |
41+
pnpm --filter @tanstack/db-ivm build
42+
pnpm --filter @tanstack/db build
43+
pnpm --filter @tanstack/electric-db-collection build
44+
pnpm --filter @tanstack/query-db-collection build
45+
46+
- name: Run Electric E2E tests
47+
run: |
48+
cd packages/electric-db-collection
49+
pnpm test:e2e
50+
env:
51+
ELECTRIC_URL: http://localhost:3000
52+
POSTGRES_HOST: localhost
53+
POSTGRES_PORT: 54321
54+
POSTGRES_USER: postgres
55+
POSTGRES_PASSWORD: password
56+
POSTGRES_DB: e2e_test
57+
58+
- name: Run Query E2E tests
59+
run: |
60+
cd packages/query-db-collection
61+
pnpm test:e2e
62+
env:
63+
ELECTRIC_URL: http://localhost:3000
64+
65+
- name: Stop Docker services
66+
if: always()
67+
run: |
68+
cd packages/db-collection-e2e/docker
69+
docker compose down -v
70+
71+
- name: Upload test results
72+
if: failure()
73+
uses: actions/upload-artifact@v4
74+
with:
75+
name: test-results
76+
path: packages/db-collection-e2e/junit/
77+
retention-days: 7

‎docs/reference/functions/createCollection.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ The utilities record type
3939

4040
#### options
4141

42-
[`CollectionConfig`](../../interfaces/CollectionConfig.md)\<[`InferSchemaOutput`](../../type-aliases/InferSchemaOutput.md)\<`T`\>, `TKey`, `T`, [`UtilsRecord`](../../type-aliases/UtilsRecord.md)\> & `object` & [`NonSingleResult`](../../type-aliases/NonSingleResult.md)
42+
[`CollectionConfig`](../../interfaces/CollectionConfig.md)\<[`InferSchemaOutput`](../../type-aliases/InferSchemaOutput.md)\<`T`\>, `TKey`, `T`, `TUtils`\> & `object` & [`NonSingleResult`](../../type-aliases/NonSingleResult.md)
4343

4444
Collection options with optional utilities
4545

@@ -148,7 +148,7 @@ The utilities record type
148148

149149
#### options
150150

151-
[`CollectionConfig`](../../interfaces/CollectionConfig.md)\<[`InferSchemaOutput`](../../type-aliases/InferSchemaOutput.md)\<`T`\>, `TKey`, `T`, [`UtilsRecord`](../../type-aliases/UtilsRecord.md)\> & `object` & [`SingleResult`](../../type-aliases/SingleResult.md)
151+
[`CollectionConfig`](../../interfaces/CollectionConfig.md)\<[`InferSchemaOutput`](../../type-aliases/InferSchemaOutput.md)\<`T`\>, `TKey`, `T`, `TUtils`\> & `object` & [`SingleResult`](../../type-aliases/SingleResult.md)
152152

153153
Collection options with optional utilities
154154

@@ -257,7 +257,7 @@ The utilities record type
257257

258258
#### options
259259

260-
[`CollectionConfig`](../../interfaces/CollectionConfig.md)\<`T`, `TKey`, `never`, [`UtilsRecord`](../../type-aliases/UtilsRecord.md)\> & `object` & [`NonSingleResult`](../../type-aliases/NonSingleResult.md)
260+
[`CollectionConfig`](../../interfaces/CollectionConfig.md)\<`T`, `TKey`, `never`, `TUtils`\> & `object` & [`NonSingleResult`](../../type-aliases/NonSingleResult.md)
261261

262262
Collection options with optional utilities
263263

@@ -366,7 +366,7 @@ The utilities record type
366366

367367
#### options
368368

369-
[`CollectionConfig`](../../interfaces/CollectionConfig.md)\<`T`, `TKey`, `never`, [`UtilsRecord`](../../type-aliases/UtilsRecord.md)\> & `object` & [`SingleResult`](../../type-aliases/SingleResult.md)
369+
[`CollectionConfig`](../../interfaces/CollectionConfig.md)\<`T`, `TKey`, `never`, `TUtils`\> & `object` & [`SingleResult`](../../type-aliases/SingleResult.md)
370370

371371
Collection options with optional utilities
372372

‎docs/reference/functions/extractSimpleComparisons.md‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,13 +9,15 @@ title: extractSimpleComparisons
99
function extractSimpleComparisons(expr): SimpleComparison[];
1010
```
1111

12-
Defined in: [packages/db/src/query/expression-helpers.ts:323](https://github.com/TanStack/db/blob/main/packages/db/src/query/expression-helpers.ts#L323)
12+
Defined in: [packages/db/src/query/expression-helpers.ts:327](https://github.com/TanStack/db/blob/main/packages/db/src/query/expression-helpers.ts#L327)
1313

1414
Extracts all simple comparisons from a WHERE expression.
1515
This is useful for simple APIs that only support basic filters.
1616

17-
Note: This only works for simple AND-ed conditions. Throws an error if it encounters
18-
unsupported operations like OR, NOT, or complex nested expressions.
17+
Note: This only works for simple AND-ed conditions and NOT-wrapped comparisons.
18+
Throws an error if it encounters unsupported operations like OR or complex nested expressions.
19+
20+
NOT operators are flattened by prefixing the operator name (e.g., `not(eq(...))` becomes `not_eq`).
1921

2022
## Parameters
2123

@@ -33,14 +35,16 @@ Array of simple comparisons
3335

3436
## Throws
3537

36-
Error if expression contains OR, NOT, or other unsupported operations
38+
Error if expression contains OR or other unsupported operations
3739

3840
## Example
3941

4042
```typescript
4143
const comparisons = extractSimpleComparisons(where)
4244
// Returns: [
4345
// { field: ['category'], operator: 'eq', value: 'electronics' },
44-
// { field: ['price'], operator: 'lt', value: 100 }
46+
// { field: ['price'], operator: 'lt', value: 100 },
47+
// { field: ['email'], operator: 'isNull' }, // No value for null checks
48+
// { field: ['status'], operator: 'not_eq', value: 'archived' }
4549
// ]
4650
```

‎docs/reference/functions/localStorageCollectionOptions.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ title: localStorageCollectionOptions
88
## Call Signature
99

1010
```ts
11-
function localStorageCollectionOptions<T, TKey>(config): CollectionConfig<InferSchemaOutput<T>, TKey, T, UtilsRecord> & object;
11+
function localStorageCollectionOptions<T, TKey>(config): CollectionConfig<InferSchemaOutput<T>, TKey, T, LocalStorageCollectionUtils> & object;
1212
```
1313

1414
Defined in: [packages/db/src/local-storage.ts:279](https://github.com/TanStack/db/blob/main/packages/db/src/local-storage.ts#L279)
@@ -52,7 +52,7 @@ Configuration options for the localStorage collection
5252

5353
### Returns
5454

55-
[`CollectionConfig`](../../interfaces/CollectionConfig.md)\<[`InferSchemaOutput`](../../type-aliases/InferSchemaOutput.md)\<`T`\>, `TKey`, `T`, [`UtilsRecord`](../../type-aliases/UtilsRecord.md)\> & `object`
55+
[`CollectionConfig`](../../interfaces/CollectionConfig.md)\<[`InferSchemaOutput`](../../type-aliases/InferSchemaOutput.md)\<`T`\>, `TKey`, `T`, [`LocalStorageCollectionUtils`](../../interfaces/LocalStorageCollectionUtils.md)\> & `object`
5656

5757
Collection options with utilities including clearStorage, getStorageSize, and acceptMutations
5858

@@ -123,10 +123,10 @@ await tx.commit()
123123
## Call Signature
124124

125125
```ts
126-
function localStorageCollectionOptions<T, TKey>(config): CollectionConfig<T, TKey, never, UtilsRecord> & object;
126+
function localStorageCollectionOptions<T, TKey>(config): CollectionConfig<T, TKey, never, LocalStorageCollectionUtils> & object;
127127
```
128128

129-
Defined in: [packages/db/src/local-storage.ts:294](https://github.com/TanStack/db/blob/main/packages/db/src/local-storage.ts#L294)
129+
Defined in: [packages/db/src/local-storage.ts:299](https://github.com/TanStack/db/blob/main/packages/db/src/local-storage.ts#L299)
130130

131131
Creates localStorage collection options for use with a standard Collection
132132

@@ -167,7 +167,7 @@ Configuration options for the localStorage collection
167167

168168
### Returns
169169

170-
[`CollectionConfig`](../../interfaces/CollectionConfig.md)\<`T`, `TKey`, `never`, [`UtilsRecord`](../../type-aliases/UtilsRecord.md)\> & `object`
170+
[`CollectionConfig`](../../interfaces/CollectionConfig.md)\<`T`, `TKey`, `never`, [`LocalStorageCollectionUtils`](../../interfaces/LocalStorageCollectionUtils.md)\> & `object`
171171

172172
Collection options with utilities including clearStorage, getStorageSize, and acceptMutations
173173

‎docs/reference/functions/parseLoadSubsetOptions.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ title: parseLoadSubsetOptions
99
function parseLoadSubsetOptions(options): object;
1010
```
1111

12-
Defined in: [packages/db/src/query/expression-helpers.ts:420](https://github.com/TanStack/db/blob/main/packages/db/src/query/expression-helpers.ts#L420)
12+
Defined in: [packages/db/src/query/expression-helpers.ts:499](https://github.com/TanStack/db/blob/main/packages/db/src/query/expression-helpers.ts#L499)
1313

1414
Convenience function to get all LoadSubsetOptions in a pre-parsed format.
1515
Good starting point for simple use cases.

‎docs/reference/interfaces/SimpleComparison.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -31,10 +31,10 @@ Defined in: [packages/db/src/query/expression-helpers.ts:46](https://github.com/
3131

3232
***
3333

34-
### value
34+
### value?
3535

3636
```ts
37-
value: any;
37+
optional value: any;
3838
```
3939

4040
Defined in: [packages/db/src/query/expression-helpers.ts:47](https://github.com/TanStack/db/blob/main/packages/db/src/query/expression-helpers.ts#L47)

0 commit comments

Comments
 (0)