From 06a95597e9f5d8c21e7681bd7cedef01bfacdcc2 Mon Sep 17 00:00:00 2001 From: David Rinaldo Date: Sat, 4 Jul 2026 11:42:06 -0400 Subject: [PATCH] chunkenc Bitstream primitives (bstream) and XOR encoder/decoder following the Prometheus TSDB variant of the Gorilla scheme. Delta-of-delta timestamps with 14/17/20/64-bit buckets, XOR-compressed float64 values. Decoder is total: arbitrary bytes produce values or ErrShortStream. --- LICENSE.md | 202 ++++++++++++++++ go.mod | 11 + go.sum | 10 + internal/chunkenc/bstream.go | 144 +++++++++++ internal/chunkenc/bstream_test.go | 164 +++++++++++++ internal/chunkenc/xor.go | 305 ++++++++++++++++++++++++ internal/chunkenc/xor_test.go | 383 ++++++++++++++++++++++++++++++ third_party/prometheus/LICENSE.md | 201 ++++++++++++++++ 8 files changed, 1420 insertions(+) create mode 100644 LICENSE.md create mode 100644 go.mod create mode 100644 go.sum create mode 100644 internal/chunkenc/bstream.go create mode 100644 internal/chunkenc/bstream_test.go create mode 100644 internal/chunkenc/xor.go create mode 100644 internal/chunkenc/xor_test.go create mode 100644 third_party/prometheus/LICENSE.md diff --git a/LICENSE.md b/LICENSE.md new file mode 100644 index 0000000..7a4a3ea --- /dev/null +++ b/LICENSE.md @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. \ No newline at end of file diff --git a/go.mod b/go.mod new file mode 100644 index 0000000..f971dfd --- /dev/null +++ b/go.mod @@ -0,0 +1,11 @@ +module git.dvdt.dev/david/ingot + +go 1.26.1 + +require github.com/stretchr/testify v1.11.1 + +require ( + github.com/davecgh/go-spew v1.1.1 // indirect + github.com/pmezard/go-difflib v1.0.0 // indirect + gopkg.in/yaml.v3 v3.0.1 // indirect +) diff --git a/go.sum b/go.sum new file mode 100644 index 0000000..c4c1710 --- /dev/null +++ b/go.sum @@ -0,0 +1,10 @@ +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/internal/chunkenc/bstream.go b/internal/chunkenc/bstream.go new file mode 100644 index 0000000..e721bd8 --- /dev/null +++ b/internal/chunkenc/bstream.go @@ -0,0 +1,144 @@ +// Chunk encoding follows Prometheus tsdb/chunkenc. See /NOTICE.md. +package chunkenc + +import ( + "errors" +) + +var ErrShortStream = errors.New("chunkenc: unexpected end of stream") + +type bstream struct { + stream []byte + count uint8 +} + +func (b *bstream) bytes() []byte { + return b.stream +} + +func (b *bstream) writeBit(bit bool) { + if b.count == 0 { + b.stream = append(b.stream, 0) + b.count = 8 + } + i := len(b.stream) - 1 + if bit { + b.stream[i] |= 1 << (b.count - 1) + } + b.count-- +} + +func (b *bstream) writeBits(u uint64, nbits int) { + // Byte-aligned fast path from the top of the value down. + u <<= 64 - uint(nbits) + for nbits >= 8 { + b.writeByte(byte(u >> 56)) + u <<= 8 + nbits -= 8 + } + for nbits > 0 { + b.writeBit((u >> 63) == 1) + u <<= 1 + nbits-- + } +} + +func (b *bstream) writeByte(byt byte) { + if b.count == 0 { + b.stream = append(b.stream, 0) + b.count = 8 + } + i := len(b.stream) - 1 + + // Split across the partial last byte and a fresh one. + b.stream[i] |= byt >> (8 - b.count) + b.stream = append(b.stream, 0) + i++ + b.stream[i] = byt << b.count + // count unchanged: same number of free bits, now in the new last byte +} + +type bstreamReader struct { + stream []byte + off int // byte offset of next unread byte + buf uint64 // bit buffer, MSB-first + valid uint8 //valid bits remaining in buf +} + +func newBReader(b []byte) bstreamReader { + return bstreamReader{stream: b} +} + +// loadNextBuffer refills buf with up to 8 bytes. Returns false at EOF. +func (r *bstreamReader) loadNextBuffer(nbits uint8) bool { + if r.off >= len(r.stream) { + return false + } + + // Fast path: 8 full bytes available. + if r.off+8 <= len(r.stream) { + r.buf = uint64(r.stream[r.off])<<56 | + uint64(r.stream[r.off+1])<<48 | + uint64(r.stream[r.off+2])<<40 | + uint64(r.stream[r.off+3])<<32 | + uint64(r.stream[r.off+4])<<24 | + uint64(r.stream[r.off+5])<<16 | + uint64(r.stream[r.off+6])<<8 | + uint64(r.stream[r.off+7]) + r.off += 8 + r.valid = 64 + return true + } + + // Tail: load what's left, left-aligned. + n := len(r.stream) - r.off + r.buf = 0 + for i := 0; i < n; i++ { + r.buf |= uint64(r.stream[r.off+i]) << (56 - 8*uint(i)) + } + r.off += n + r.valid = uint8(n * 8) + return true +} + +func (r *bstreamReader) readBit() (bool, error) { + if r.valid == 0 { + if !r.loadNextBuffer(1) { + return false, ErrShortStream + } + } + bit := r.buf&(1<<63) != 0 + r.buf <<= 1 + r.valid-- + return bit, nil +} + +func (r *bstreamReader) readBits(nbits int) (uint64, error) { + if nbits == 0 { + return 0, nil + } + var v uint64 + remaining := uint8(nbits) + + for remaining > 0 { + if r.valid == 0 { + if !r.loadNextBuffer(remaining) { + return 0, ErrShortStream + } + } + take := remaining + if take > r.valid { + take = r.valid + } + v = (v << take) | (r.buf >> (64 - take)) + r.buf <<= take + r.valid -= take + remaining -= take + } + return v, nil +} + +func (r *bstreamReader) readByte() (byte, error) { + v, err := r.readBits(8) + return byte(v), err +} diff --git a/internal/chunkenc/bstream_test.go b/internal/chunkenc/bstream_test.go new file mode 100644 index 0000000..b9a41c6 --- /dev/null +++ b/internal/chunkenc/bstream_test.go @@ -0,0 +1,164 @@ +package chunkenc + +import ( + "fmt" + "math/rand" + "testing" + + "github.com/stretchr/testify/assert" +) + +// bstreamBytes writes (value, nbits) pairs and returns the encoded bytes. +func bstreamBytes(pairs [][2]uint64) []byte { + var b bstream + for _, p := range pairs { + b.writeBits(p[0], int(p[1])) + } + return b.bytes() +} + +func TestBstream(t *testing.T) { + rnd := rand.New(rand.NewSource(42)) + + type readOp struct { + nbits int + wantVal uint64 + wantErr error + } + + ok := func(val uint64, nbits int) readOp { return readOp{nbits, val, nil} } + short := func(nbits int) readOp { return readOp{nbits, 0, ErrShortStream} } + + tests := []struct { + name string + data []byte + reads []readOp + }{ + // Error paths: raw bytes, reads that exceed the data. + {name: "nil_read_1", data: nil, reads: []readOp{short(1)}}, + {name: "nil_read_64", data: nil, reads: []readOp{short(64)}}, + {name: "nil_read_0", data: nil, reads: []readOp{ok(0, 0)}}, + {name: "1_byte_read_16", data: []byte{0xFF}, reads: []readOp{short(16)}}, + {name: "7_bytes_read_64", data: []byte{1, 2, 3, 4, 5, 6, 7}, reads: []readOp{short(64)}}, + {name: "exact_byte_then_past_end", data: []byte{0xAB}, reads: []readOp{ok(0xAB, 8), short(1)}}, + {name: "two_bytes_then_past_end", data: []byte{0xAB, 0xCD}, reads: []readOp{ok(0xAB, 8), ok(0xCD, 8), short(1)}}, + {name: "partial_bits_then_short", data: []byte{0xFF}, reads: []readOp{ok(0b1111, 4), short(8)}}, + {name: "8_bytes_exact_64", data: []byte{0xDE, 0xAD, 0xBE, 0xEF, 0xCA, 0xFE, 0xBA, 0xBE}, reads: []readOp{ok(0xDEADBEEFCAFEBABE, 64), short(1)}}, + + // Round-trip: write via bstream, read back. + {name: "zero_width", data: bstreamBytes([][2]uint64{{0, 0}}), reads: []readOp{ok(0, 0)}}, + {name: "single_bit_true", data: bstreamBytes([][2]uint64{{1, 1}}), reads: []readOp{ok(1, 1)}}, + {name: "single_bit_false", data: bstreamBytes([][2]uint64{{0, 1}}), reads: []readOp{ok(0, 1)}}, + {name: "single_byte", data: bstreamBytes([][2]uint64{{0xAB, 8}}), reads: []readOp{ok(0xAB, 8)}}, + {name: "full_64_bits", data: bstreamBytes([][2]uint64{{0xDEADBEEFCAFEBABE, 64}}), reads: []readOp{ok(0xDEADBEEFCAFEBABE, 64)}}, + {name: "all_ones_8", data: bstreamBytes([][2]uint64{{0xFF, 8}}), reads: []readOp{ok(0xFF, 8)}}, + {name: "all_zeros_8", data: bstreamBytes([][2]uint64{{0, 8}}), reads: []readOp{ok(0, 8)}}, + {name: "all_ones_64", data: bstreamBytes([][2]uint64{{^uint64(0), 64}}), reads: []readOp{ok(^uint64(0), 64)}}, + {name: "all_zeros_64", data: bstreamBytes([][2]uint64{{0, 64}}), reads: []readOp{ok(0, 64)}}, + { + name: "byte_aligned_sequence", + data: bstreamBytes([][2]uint64{{0xAA, 8}, {0xBB, 8}, {0xCC, 8}}), + reads: []readOp{ok(0xAA, 8), ok(0xBB, 8), ok(0xCC, 8)}, + }, + { + name: "non_aligned_crossing_boundary", + data: bstreamBytes([][2]uint64{{0b101, 3}, {0b11001, 5}, {0xFF, 8}, {1, 1}}), + reads: []readOp{ok(0b101, 3), ok(0b11001, 5), ok(0xFF, 8), ok(1, 1)}, + }, + { + name: "alternating_single_bits", + data: bstreamBytes([][2]uint64{{1, 1}, {0, 1}, {1, 1}, {0, 1}, {1, 1}, {0, 1}, {1, 1}, {0, 1}}), + reads: []readOp{ + ok(1, 1), ok(0, 1), ok(1, 1), ok(0, 1), + ok(1, 1), ok(0, 1), ok(1, 1), ok(0, 1), + }, + }, + { + name: "sequential_64_bit_writes", + data: bstreamBytes([][2]uint64{{0x0123456789ABCDEF, 64}, {0xFEDCBA9876543210, 64}, {0, 64}}), + reads: []readOp{ok(0x0123456789ABCDEF, 64), ok(0xFEDCBA9876543210, 64), ok(0, 64)}, + }, + { + name: "every_width_1_through_64", + data: func() []byte { + pairs := make([][2]uint64, 64) + for i := range pairs { + nbits := i + 1 + val := uint64(1) + if nbits > 1 { + val |= uint64(1) << (nbits - 1) + } + pairs[i] = [2]uint64{val, uint64(nbits)} + } + return bstreamBytes(pairs) + }(), + reads: func() []readOp { + ops := make([]readOp, 64) + for i := range ops { + nbits := i + 1 + val := uint64(1) + if nbits > 1 { + val |= uint64(1) << (nbits - 1) + } + ops[i] = ok(val, nbits) + } + return ops + }(), + }, + { + name: "high_bits_ignored", + data: bstreamBytes([][2]uint64{{0xFFFFFFFFFFFFFFFF, 1}, {0xFFFFFFFFFFFFFFFF, 4}, {0xFFFFFFFFFFFFFFFF, 8}}), + reads: []readOp{ok(1, 1), ok(0xF, 4), ok(0xFF, 8)}, + }, + { + name: "mixed_widths", + data: bstreamBytes([][2]uint64{{0b1, 1}, {0xABCD, 16}, {0, 1}, {0xFF, 8}, {0b110, 3}, {0xDEADBEEF, 32}, {1, 1}, {0, 0}}), + reads: []readOp{ok(0b1, 1), ok(0xABCD, 16), ok(0, 1), ok(0xFF, 8), ok(0b110, 3), ok(0xDEADBEEF, 32), ok(1, 1), ok(0, 0)}, + }, + } + + // Random round-trip cases. + for i := 0; i < 100; i++ { + n := rnd.Intn(200) + 1 + pairs := make([][2]uint64, n) + reads := make([]readOp, n) + for j := range pairs { + nbits := rnd.Intn(64) + 1 + val := rnd.Uint64() & (^uint64(0) >> (64 - uint(nbits))) + pairs[j] = [2]uint64{val, uint64(nbits)} + reads[j] = ok(val, nbits) + } + tests = append(tests, struct { + name string + data []byte + reads []readOp + }{ + name: fmt.Sprintf("random_%03d", i), + data: bstreamBytes(pairs), + reads: reads, + }) + } + + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + r := newBReader(tc.data) + for i, op := range tc.reads { + got, err := r.readBits(op.nbits) + assert.Equal(t, op.wantVal, got, "op %d val (nbits=%d)", i, op.nbits) + assert.Equal(t, op.wantErr, err, "op %d err (nbits=%d)", i, op.nbits) + } + }) + } +} + +func FuzzBstreamReader(f *testing.F) { + f.Fuzz(func(t *testing.T, data []byte, nbits uint8) { + r := newBReader(data) + n := int(nbits%64) + 1 + for { + if _, err := r.readBits(n); err != nil { + break // must terminate via error, never panic + } + } + }) +} diff --git a/internal/chunkenc/xor.go b/internal/chunkenc/xor.go new file mode 100644 index 0000000..a353c96 --- /dev/null +++ b/internal/chunkenc/xor.go @@ -0,0 +1,305 @@ +// Chunk encoding follows Prometheus tsdb/chunkenc. See /NOTICE.md. +package chunkenc + +import ( + "encoding/binary" + "errors" + "math" + "math/bits" +) + +type XORChunk struct { + b bstream +} + +func NewXORChunk() *XORChunk { + return &XORChunk{b: bstream{stream: make([]byte, 2), count: 0}} +} + +func (c *XORChunk) NumSamples() int { + return int(binary.BigEndian.Uint16(c.b.bytes())) +} + +func (c *XORChunk) Appender() (*xorAppender, error) { + if c.NumSamples() > 0 { + return nil, errors.New("chunkenc: appender on non-empty chunk") + } + return &xorAppender{b: &c.b, leading: 0xff}, nil +} + +func (c *XORChunk) Bytes() []byte { + return c.b.bytes() +} + +type xorAppender struct { + b *bstream + t int64 // last timestamp + v float64 // last value + tDelta uint64 // last delta + leading uint8 // current XOR window + trailing uint8 +} + +func (a *xorAppender) Append(t int64, v float64) { + num := binary.BigEndian.Uint16(a.b.stream) + + switch num { + case 0: + // First sample: raw 64-bit timestamp and value. + a.b.writeBits(uint64(t), 64) + a.b.writeBits(math.Float64bits(v), 64) + + case 1: + // Second sample: fixed 14-bit first delta, then XOR value. + delta := uint64(t - a.t) + a.b.writeBits(delta, 14) + a.writeVDelta(v) + a.tDelta = delta + + default: + delta := uint64(t - a.t) + dod := int64(delta) - int64(a.tDelta) + + // Prefix-code ladder. Buckets match the Prometheus variant + // (14/17/20/64) rather than the paper's (7/9/12/32) - wider + // buckets tolerate millisecond timestamps and jittery sources. + switch { + case dod == 0: + a.b.writeBit(false) + case bitRange(dod, 14): + a.b.writeBits(0b10, 2) + a.b.writeBits(uint64(dod)&((1<<14)-1), 14) + case bitRange(dod, 17): + a.b.writeBits(0b110, 3) + a.b.writeBits(uint64(dod)&((1<<17)-1), 17) + case bitRange(dod, 20): + a.b.writeBits(0b1110, 4) + a.b.writeBits(uint64(dod)&((1<<20)-1), 20) + default: + a.b.writeBits(0b1111, 4) + a.b.writeBits(uint64(dod), 64) + } + + a.writeVDelta(v) + a.tDelta = delta + } + + a.t = t + a.v = v + binary.BigEndian.PutUint16(a.b.stream, num+1) +} + +// bitRange reports whether x fits in an nbits-wide two's-complement field. +func bitRange(x int64, nbits int) bool { + return -(1<<(nbits-1)) <= x && x < 1<<(nbits-1) +} + +func (a *xorAppender) writeVDelta(v float64) { + xor := math.Float64bits(v) ^ math.Float64bits(a.v) + + if xor == 0 { + a.b.writeBit(false) + return + } + a.b.writeBit(true) + + leading := uint8(bits.LeadingZeros64(xor)) + trailing := uint8(bits.TrailingZeros64(xor)) + + // Leading is stored in 5 bits; clamp so it fits. + if leading > 31 { + leading = 31 + } + + if a.leading != 0xff && leading >= a.leading && trailing >= a.trailing { + // New meaningful bits fit inside the previous window: reuse it. + a.b.writeBit(false) + a.b.writeBits(xor>>a.trailing, int(64-a.leading-a.trailing)) + return + } + + // New window. + a.leading, a.trailing = leading, trailing + a.b.writeBit(true) + a.b.writeBits(uint64(leading), 5) + + // sigbits can be 64 only when leading == trailing == 0, which can't + // happen here (xor != 0 and both counted on the same word), so the + // 6-bit field always fits... except leading was clamped, so recompute + // from the clamped values. + sigbits := 64 - int(leading) - int(trailing) + a.b.writeBits(uint64(sigbits), 6) + a.b.writeBits(xor>>trailing, sigbits) +} + +// Iterator decodes the chunk. Snapshot semantics: it reads the byte slice +// as it exists at creation; don't append concurrently. +func (c *XORChunk) Iterator() *xorIterator { + return &xorIterator{ + br: newBReader(c.b.bytes()[2:]), + total: uint16(c.NumSamples()), + } +} + +type xorIterator struct { + br bstreamReader + total uint16 + read uint16 + + t int64 + v float64 + tDelta uint64 + + leading uint8 + trailing uint8 + + err error +} + +func (it *xorIterator) At() (int64, float64) { + return it.t, it.v +} + +func (it *xorIterator) Err() error { + return it.err +} + +func (it *xorIterator) Next() bool { + if it.err != nil || it.read >= it.total { + return false + } + + switch it.read { + case 0: + t, err := it.br.readBits(64) + if err != nil { + it.err = err + return false + } + v, err := it.br.readBits(64) + if err != nil { + it.err = err + return false + } + it.t = int64(t) + it.v = math.Float64frombits(v) + + case 1: + delta, err := it.br.readBits(14) + if err != nil { + it.err = err + return false + } + it.tDelta = delta + it.t += int64(delta) + if !it.readVDelta() { + return false + } + + default: + // Walk the prefix tree: count 1-bits unitl a 0 or four 1s. + var d byte + for i := 0; i < 4; i++ { + bit, err := it.br.readBit() + if err != nil { + it.err = err + return false + } + if !bit { + break + } + d++ + } + + var dod int64 + switch d { + case 0: + //dod == 0 + case 1: + dod = it.readSigned(14) + case 2: + dod = it.readSigned(17) + case 3: + dod = it.readSigned(20) + case 4: + bits64, err := it.br.readBits(64) + if err != nil { + it.err = err + return false + } + dod = int64(bits64) + } + if it.err != nil { + return false + } + + it.tDelta = uint64(int64(it.tDelta) + dod) + it.t += int64(it.tDelta) + if !it.readVDelta() { + return false + } + } + + it.read++ + return true +} + +// readSigned reads an nbits two's-complement field and sign-extends it. +func (it *xorIterator) readSigned(nbits int) int64 { + v, err := it.br.readBits(nbits) + if err != nil { + it.err = err + return 0 + } + return int64(v<<(64-uint(nbits))) >> (64 - uint(nbits)) +} + +func (it *xorIterator) readVDelta() bool { + bit, err := it.br.readBit() + if err != nil { + it.err = err + return false + } + if !bit { + // Value unchanged. + return true + } + + bit, err = it.br.readBit() + if err != nil { + it.err = err + return false + } + + if bit { + // New window. + l, err := it.br.readBits(5) + if err != nil { + it.err = err + return false + } + s, err := it.br.readBits(6) + if err != nil { + it.err = err + return false + } + // sigbits=64 overflows the 6-bit field to 0; unwrap it. + if s == 0 { + s = 64 + } + it.leading = uint8(l) + it.trailing = uint8(64 - l - s) + } + + sigbits := int(64 - it.leading - it.trailing) + xor, err := it.br.readBits(sigbits) + if err != nil { + it.err = err + return false + } + + vbits := math.Float64bits(it.v) + vbits ^= xor << it.trailing + it.v = math.Float64frombits(vbits) + return true +} diff --git a/internal/chunkenc/xor_test.go b/internal/chunkenc/xor_test.go new file mode 100644 index 0000000..fb7316c --- /dev/null +++ b/internal/chunkenc/xor_test.go @@ -0,0 +1,383 @@ +package chunkenc + +import ( + "encoding/binary" + "errors" + "fmt" + "math" + "math/rand" + "testing" + + "github.com/stretchr/testify/assert" +) + +// encodeChunk builds a valid XOR chunk from samples and returns its bytes. +func encodeChunk(samples [][2]float64) []byte { + c := NewXORChunk() + a, _ := c.Appender() + for _, s := range samples { + a.Append(int64(s[0]), s[1]) + } + return append([]byte(nil), c.Bytes()...) +} + +func TestXORChunk(t *testing.T) { + rnd := rand.New(rand.NewSource(42)) + + type readOp struct { + wantNext bool + wantT int64 + wantVBits uint64 + } + + type testCase struct { + name string + data []byte + reads []readOp + wantIterErr error + maxBytes int + wantAppenderErr error + } + + nonEmptyErr := errors.New("chunkenc: appender on non-empty chunk") + + // successReads: one successful read per sample, then a terminal Next()=false. + successReads := func(samples [][2]float64) []readOp { + ops := make([]readOp, len(samples)+1) + for i, s := range samples { + ops[i] = readOp{true, int64(s[0]), math.Float64bits(s[1])} + } + term := readOp{wantNext: false} + if len(samples) > 0 { + last := samples[len(samples)-1] + term.wantT = int64(last[0]) + term.wantVBits = math.Float64bits(last[1]) + } + ops[len(samples)] = term + return ops + } + + // --- Build sample sets for named round-trip cases --- + + adversarialVals := []float64{ + math.NaN(), math.Inf(1), math.Inf(-1), 0, math.Copysign(0, -1), + math.MaxFloat64, math.SmallestNonzeroFloat64, -1e300, + } + adversarial := make([][2]float64, len(adversarialVals)) + for i, v := range adversarialVals { + adversarial[i] = [2]float64{float64(1000 + i*15), v} + } + + jitteryTimes := []int64{1000, 1015, 1030, 1031, 1500, 1501, 200000, 200015, 9000000000} + jittery := make([][2]float64, len(jitteryTimes)) + for i, ts := range jitteryTimes { + jittery[i] = [2]float64{float64(ts), float64(i) * 1.1} + } + + singleSample := [][2]float64{{1000, 71.3}} + repeatedValue := [][2]float64{{1000, 71.3}, {1015, 71.3}} + changedValue := [][2]float64{{1000, 71.3}, {1015, 71.4}} + traceSeq := [][2]float64{{1000, 71.3}, {1015, 71.3}, {1030, 71.4}, {1045, 71.4}} + + tests := []testCase{ + // --- Round-trip: encode then decode --- + { + name: "single_sample", + data: encodeChunk(singleSample), + reads: successReads(singleSample), + wantIterErr: nil, + maxBytes: math.MaxInt, + wantAppenderErr: nonEmptyErr, + }, + { + name: "repeated_value", + data: encodeChunk(repeatedValue), + reads: successReads(repeatedValue), + wantIterErr: nil, + maxBytes: math.MaxInt, + wantAppenderErr: nonEmptyErr, + }, + { + name: "changed_value", + data: encodeChunk(changedValue), + reads: successReads(changedValue), + wantIterErr: nil, + maxBytes: math.MaxInt, + wantAppenderErr: nonEmptyErr, + }, + { + name: "trace_sequence", + data: encodeChunk(traceSeq), + reads: successReads(traceSeq), + wantIterErr: nil, + maxBytes: 30, + wantAppenderErr: nonEmptyErr, + }, + { + name: "adversarial_values", + data: encodeChunk(adversarial), + reads: successReads(adversarial), + wantIterErr: nil, + maxBytes: math.MaxInt, + wantAppenderErr: nonEmptyErr, + }, + { + name: "negative_and_jittery_dods", + data: encodeChunk(jittery), + reads: successReads(jittery), + wantIterErr: nil, + maxBytes: math.MaxInt, + wantAppenderErr: nonEmptyErr, + }, + + // --- Error paths: truncated or empty data --- + { + name: "zero_samples", + data: []byte{0, 0}, + reads: []readOp{{false, 0, 0}}, + wantIterErr: nil, + maxBytes: math.MaxInt, + wantAppenderErr: nil, // empty chunk, Appender succeeds + }, + { + // Header claims 1 sample, no data after header. + // Case 0: readBits(64) for timestamp fails immediately. + name: "no_data_after_header", + data: []byte{0, 1}, + reads: []readOp{{false, 0, 0}}, + wantIterErr: ErrShortStream, + maxBytes: math.MaxInt, + wantAppenderErr: nonEmptyErr, + }, + { + // Header claims 1, only 4 bytes of data (partial timestamp). + // Case 0: readBits(64) for timestamp fails mid-read. + name: "truncated_first_timestamp", + data: func() []byte { + d := make([]byte, 6) // 2 header + 4 data + binary.BigEndian.PutUint16(d, 1) + return d + }(), + reads: []readOp{{false, 0, 0}}, + wantIterErr: ErrShortStream, + maxBytes: math.MaxInt, + wantAppenderErr: nonEmptyErr, + }, + { + // Header claims 1, full timestamp but only 4 bytes of value. + // Case 0: readBits(64) for value fails; it.t and it.v not set + // (assigned only after both reads succeed). + name: "truncated_first_value", + data: func() []byte { + d := make([]byte, 14) // 2 header + 8 timestamp + 4 partial value + binary.BigEndian.PutUint16(d, 1) + binary.BigEndian.PutUint64(d[2:], uint64(1000)) + return d + }(), + reads: []readOp{{false, 0, 0}}, + wantIterErr: ErrShortStream, + maxBytes: math.MaxInt, + wantAppenderErr: nonEmptyErr, + }, + { + // Header claims 2, data for exactly 1 sample (16 bytes). + // Case 1: readBits(14) for delta fails (no data remains). + // At() returns last successful sample. + name: "truncated_second_delta", + data: func() []byte { + d := make([]byte, 18) // 2 header + 16 data = exactly 1 sample + binary.BigEndian.PutUint16(d, 2) + binary.BigEndian.PutUint64(d[2:], uint64(1000)) + binary.BigEndian.PutUint64(d[10:], math.Float64bits(71.3)) + return d + }(), + reads: []readOp{ + {true, 1000, math.Float64bits(71.3)}, + {false, 1000, math.Float64bits(71.3)}, + }, + wantIterErr: ErrShortStream, + maxBytes: math.MaxInt, + wantAppenderErr: nonEmptyErr, + }, + { + // Header claims 2. Data: 1 valid sample + 14-bit delta + value-changed(1) + + // new-window(1), then truncated before leading-zeros field. + // Case 1: delta succeeds (it.t updated), readVDelta fails inside + // new-window branch at readBits(5). + name: "truncated_second_vdelta", + data: func() []byte { + d := make([]byte, 20) // 2 header + 18 data bytes (144 bits) + binary.BigEndian.PutUint16(d, 2) + binary.BigEndian.PutUint64(d[2:], uint64(1000)) + binary.BigEndian.PutUint64(d[10:], math.Float64bits(71.3)) + // Bits 128-141: 14-bit delta = 15 (0b00000000001111) + // Byte 18 (bits 128-135): 0x00 + // Byte 19 (bits 136-143): + // 136-141 = bottom 6 bits of delta (001111) + // 142 = value-changed (1) + // 143 = new-window (1) + // = 0b00111111 = 0x3F + d[19] = 0x3F + return d + }(), + reads: []readOp{ + {true, 1000, math.Float64bits(71.3)}, + {false, 1015, math.Float64bits(71.3)}, // t advanced, v unchanged + }, + wantIterErr: ErrShortStream, + maxBytes: math.MaxInt, + wantAppenderErr: nonEmptyErr, + }, + { + // Header claims 3. Data: 2 valid samples (same value, so sample 1 + // is 14-bit delta + 1-bit value-unchanged = 15 bits). Total data = + // 143 bits in 18 bytes (1 padding bit). Sample 2 DoD prefix reads + // the padding zero (dod=0, t advances), then readVDelta fails. + name: "truncated_third_vdelta", + data: func() []byte { + d := make([]byte, 20) // 2 header + 18 data (144 bits) + binary.BigEndian.PutUint16(d, 3) + binary.BigEndian.PutUint64(d[2:], uint64(1000)) + binary.BigEndian.PutUint64(d[10:], math.Float64bits(71.3)) + // Bits 128-141: 14-bit delta = 15 + // Bit 142: value-unchanged (0) + // Bit 143: padding (0) + // Byte 18 = 0x00, Byte 19 = 0b00111100 = 0x3C + d[19] = 0x3C + return d + }(), + reads: []readOp{ + {true, 1000, math.Float64bits(71.3)}, + {true, 1015, math.Float64bits(71.3)}, + {false, 1030, math.Float64bits(71.3)}, // dod=0 decoded from padding, vdelta fails + }, + wantIterErr: ErrShortStream, + maxBytes: math.MaxInt, + wantAppenderErr: nonEmptyErr, + }, + } + + // Random walk round-trip cases. + for i := 0; i < 100; i++ { + var samples [][2]float64 + ts, v := int64(rnd.Intn(1e6)), rnd.Float64()*100 + for j := 0; j < 120; j++ { + samples = append(samples, [2]float64{float64(ts), v}) + ts += 15000 + int64(rnd.Intn(100)) - 50 + v += rnd.Float64() - 0.5 + } + tests = append(tests, testCase{ + name: fmt.Sprintf("random_walk_%03d", i), + data: encodeChunk(samples), + reads: successReads(samples), + wantIterErr: nil, + maxBytes: math.MaxInt, + wantAppenderErr: nonEmptyErr, + }) + } + + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + c := &XORChunk{b: bstream{stream: tc.data}} + + it := c.Iterator() + for i, r := range tc.reads { + next := it.Next() + gotT, gotV := it.At() + assert.Equal(t, r.wantNext, next, "read %d Next()", i) + assert.Equal(t, r.wantT, gotT, "read %d timestamp", i) + assert.Equal(t, r.wantVBits, math.Float64bits(gotV), "read %d value", i) + } + assert.Equal(t, tc.wantIterErr, it.Err()) + + assert.LessOrEqual(t, len(tc.data), tc.maxBytes) + + _, appErr := c.Appender() + assert.Equal(t, tc.wantAppenderErr, appErr) + }) + } +} + +func FuzzXORIterator(f *testing.F) { + c := NewXORChunk() + a, _ := c.Appender() + a.Append(1000, 71.3) + a.Append(1015, 71.4) + f.Add(c.Bytes()) + + f.Fuzz(func(t *testing.T, data []byte) { + if len(data) < 2 { + return + } + chunk := &XORChunk{b: bstream{stream: data}} + it := chunk.Iterator() + for it.Next() { + } + // Termination without panic is the only assertion. + }) +} + +// benchChunk fills a chunk with 120 samples from gen and returns it. +func benchChunk(gen func(i int) float64) *XORChunk { + c := NewXORChunk() + a, _ := c.Appender() + ts := int64(0) + for i := 0; i < 120; i++ { + a.Append(ts, gen(i)) + ts += 15000 + } + return c +} + +func BenchmarkAppend(b *testing.B) { + rnd := rand.New(rand.NewSource(42)) + vals := make([]float64, 120) + v := 70.0 + for i := range vals { + vals[i] = v + v += rnd.Float64() - 0.5 + } + + b.ReportAllocs() + b.ResetTimer() + for i := 0; i < b.N; i++ { + c := NewXORChunk() + a, _ := c.Appender() + ts := int64(0) + for j := 0; j < 120; j++ { + a.Append(ts, vals[j]) + ts += 15000 + } + } + b.ReportMetric(float64(b.N*120)/b.Elapsed().Seconds(), "appends/sec") +} + +func TestBytesPerSample(t *testing.T) { + rnd := rand.New(rand.NewSource(42)) + + tests := []struct { + name string + max float64 + gen func(i int) float64 + }{ + {"constant", 0.5, func(i int) float64 { return 71.3 }}, + {"stepped_sensor", 3.0, func(i int) float64 { + return 70 + math.Floor(float64(i)/8)*0.1 + }}, + {"integer_counter", 3.0, func(i int) float64 { + return float64(1000 + i*3) + }}, + {"full_precision_walk", 10.0, func(i int) float64 { + return 70 + rnd.NormFloat64() + }}, + } + + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + c := benchChunk(tc.gen) + bps := float64(len(c.Bytes())) / 120.0 + t.Logf("%s: %.3f bytes/sample (%d bytes total)", tc.name, bps, len(c.Bytes())) + assert.LessOrEqual(t, bps, tc.max, "%s bytes/sample exceeds ceiling", tc.name) + }) + } +} diff --git a/third_party/prometheus/LICENSE.md b/third_party/prometheus/LICENSE.md new file mode 100644 index 0000000..f49a4e1 --- /dev/null +++ b/third_party/prometheus/LICENSE.md @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. \ No newline at end of file