metadata: implement write using encode, not vice versa

Using write() for encode() is slower due to the use of BytesIO,
and we never really stream the data out anyway since it is part
of an object that we build, so it's all in memory.

Thus, implement write() using encode() rather than the other
way around.

Together with the previous patches, this speeds up encoding the
metadata by about 54%, and indexing by about 15-20% (my system goes
from ~16.3k paths/sec to ~19.7k paths/sec when all the filesystem
data is already buffered in memory).

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>

diff --git a/lib/bup/metadata.py b/lib/bup/metadata.py
index 0a7514d4d78181c2c82fcb4f9adcd97a65820c96..d799cc4b645ad94db474846c3adedc5beab0d44e 100644 (file)
--- a/lib/bup/metadata.py
+++ b/lib/bup/metadata.py
@@ -782,6 +782,10 @@ class Metadata:
         return ''.join(result)
 
     def write(self, port, include_path=True):
+        port.write(self.encode(include_path=include_path))
+
+    def encode(self, include_path=True):
+        ret = []
         records = include_path and [(_rec_tag_path, self._encode_path())] or []
         records.extend([(_rec_tag_common_v3, self._encode_common()),
                         (_rec_tag_symlink_target,
@@ -793,14 +797,10 @@ class Metadata:
                         (_rec_tag_linux_xattr, self._encode_linux_xattr())])
         for tag, data in records:
             if data:
-                vint.write_vuint(port, tag)
-                vint.write_bvec(port, data)
-        vint.write_vuint(port, _rec_tag_end)
-
-    def encode(self, include_path=True):
-        port = BytesIO()
-        self.write(port, include_path)
-        return port.getvalue()
+                ret.extend((vint.encode_vuint(tag),
+                            vint.encode_bvec(data)))
+        ret.append(vint.encode_vuint(_rec_tag_end))
+        return b''.join(ret)
 
     def copy(self):
         return deepcopy(self)