terraform/lang
Martin Atkins 70fed23be5 lang/funcs: File hashing functions stream data from disk
Previously our file hashing functions were backed by the same "read file
into memory" function we use for situations like "file" and "templatefile",
meaning that they'd read the entire file into memory first and then
calculate the hash from that buffer.

All of the hash implementations we use here can calculate hashes from a
sequence of smaller buffer writes though, so there's no actual need for
us to create a file-sized temporary buffer here.

This, then, is a small refactoring of our underlying function into two
parts, where one is responsible for deciding the actual filename to load
opening it, and the other is responsible for buffering the file into
memory. Our hashing functions can then use only the first function and
skip the second.

This then allows us to use io.Copy to stream from the file into the
hashing function in smaller chunks, possibly of a size chosen by the hash
function if it happens to implement io.ReaderFrom.

The new implementation is functionally equivalent to the old but should
use less temporary memory if the user passes a large file to one of the
hashing functions.
2021-05-12 09:28:31 -07:00
..
blocktoattr make blocktoattr an hcldec.UnknownBody 2021-04-13 18:42:15 -04:00
funcs lang/funcs: File hashing functions stream data from disk 2021-05-12 09:28:31 -07:00
testdata/functions-test lang/funcs: Update fileset() function to include path as separate first argument, automatically trim the path argument from results, and ensure results are always canonical with forward slash path separators 2019-08-30 20:19:44 -04:00
data.go remove GetModuleInstanceOutput 2020-04-14 14:49:10 -04:00
data_test.go eval Data needs to operate on whole modules 2020-04-12 10:50:31 -04:00
doc.go lang: new package for the runtime parts of the config language 2018-10-16 18:44:26 -07:00
eval.go Fix diags non-assignment bugs 2020-11-20 13:37:23 -05:00
eval_test.go allow path and terraform in self-block eval 2020-11-02 14:00:58 -05:00
functions.go lang/funcs: add (console-only) TypeFunction (#28501) 2021-04-23 10:29:50 -04:00
functions_test.go lang/funcs: "one" function 2021-04-12 15:32:03 -07:00
references.go vendor: switch to HCL 2.0 in the HCL repository 2019-10-02 15:10:21 -07:00
scope.go lang/funcs: add (console-only) TypeFunction (#28501) 2021-04-23 10:29:50 -04:00