2014-09-16 Geoffrey Garen bmalloc: moved line caches from the deallocator to the allocator https://bugs.webkit.org/show_bug.cgi?id=136868 Reviewed by Gavin Barraclough. I did this mostly as a simplification, to make it easier to change the allocation strategy. No throughput change on MallocBench. Saves about 50kB. Since the deallocator needs to lock the heap when freeing lines anyway, there isn't much benefit to giving the deallocator a local cache of deallocated lines. We still give the allocator a local cache of lines because that does reduce the frequency at which it needs to lock the heap in order to acquire more lines. * bmalloc/Allocator.cpp: (bmalloc::Allocator::scavenge): (bmalloc::Allocator::allocateSmallLine): (bmalloc::Allocator::allocateMediumLine): (bmalloc::Allocator::allocateMedium): (bmalloc::Allocator::allocateSlowCase): * bmalloc/Allocator.h: * bmalloc/Deallocator.cpp: (bmalloc::Deallocator::Deallocator): (bmalloc::Deallocator::scavenge): (bmalloc::Deallocator::processObjectLog): (bmalloc::Deallocator::deallocateSmallLine): Deleted. (bmalloc::Deallocator::allocateSmallLine): Deleted. (bmalloc::Deallocator::deallocateMediumLine): Deleted. (bmalloc::Deallocator::allocateMediumLine): Deleted. * bmalloc/Deallocator.h: * bmalloc/Sizes.h: * bmalloc/VMAllocate.h: Took the opportunity to make the line cache size exactly one page in size. That's about what we were shooting for anyway, and it may make it easier to switch to per-page allocation in future. 2014-09-15 Geoffrey Garen bmalloc: allocate small and medium objects using the same bump pointer class https://bugs.webkit.org/show_bug.cgi?id=136843 Reviewed by Gavin Barraclough. 4% speedup on MallocBench. Now that medium-sized objects have dedicated per-size allocators, they don't need to use an arbitrary bump pointer allocator. This means that every allocator knows how many objects it will allocate from the start, and we don't need a post-processing step to adjust refcounts based on real allocation count. * bmalloc.xcodeproj/project.pbxproj: Renamed SmallAllocator to BumpAllocator since it's used for small and medium objects now. * bmalloc/Allocator.cpp: (bmalloc::Allocator::Allocator): Updated to use new interface. (bmalloc::Allocator::scavenge): To "retire" an allocator, we just need to make sure that we finish allocating all the objects in it. (bmalloc::Allocator::allocateMedium): (bmalloc::Allocator::allocateSlowCase): (bmalloc::Allocator::retire): Deleted. (bmalloc::Allocator::processSmallAllocatorLog): Deleted. (bmalloc::Allocator::processMediumAllocatorLog): Deleted. * bmalloc/Allocator.h: (bmalloc::Allocator::allocateFastCase): Removed abstractions and data used to post-process an allocator based on how many objects it allocated. * bmalloc/BumpAllocator.h: Copied from Source/bmalloc/bmalloc/SmallAllocator.h. (bmalloc::BumpAllocator::BumpAllocator): (bmalloc::BumpAllocator::init): (bmalloc::BumpAllocator::line): (bmalloc::BumpAllocator::validate): (bmalloc::BumpAllocator::allocate): (bmalloc::BumpAllocator::refill): (bmalloc::BumpAllocator::clear): Updated these functions to be agnostic about the kinds of lines they allocate into. In some cases, the line type must be provided as a template parameter by the caller. (bmalloc::SmallAllocator::SmallAllocator): Deleted. (bmalloc::SmallAllocator::line): Deleted. (bmalloc::SmallAllocator::allocate): Deleted. (bmalloc::SmallAllocator::objectCount): Deleted. (bmalloc::SmallAllocator::derefCount): Deleted. (bmalloc::SmallAllocator::refill): Deleted. (bmalloc::SmallAllocator::clear): Deleted. * bmalloc/ObjectType.h: (bmalloc::isMedium): * bmalloc/SmallAllocator.h: (bmalloc::SmallAllocator::isNull): Deleted. (bmalloc::SmallAllocator::canAllocate): Deleted. (bmalloc::SmallAllocator::SmallAllocator): Deleted. (bmalloc::SmallAllocator::line): Deleted. (bmalloc::SmallAllocator::allocate): Deleted. (bmalloc::SmallAllocator::objectCount): Deleted. (bmalloc::SmallAllocator::derefCount): Deleted. (bmalloc::SmallAllocator::refill): Deleted. (bmalloc::SmallAllocator::clear): Deleted. 2014-09-12 Geoffrey Garen Fixed a goof in bmalloc Vector sizing https://bugs.webkit.org/show_bug.cgi?id=136795 Reviewed by Gavin Barraclough and Sam Weinig. We want our minimum vector to be page-sized since the OS will give us a page no matter what -- but we want that many bytes, and not enough bytes to store that many elements. * bmalloc/Vector.h: Math is hard. 2014-09-11 Geoffrey Garen bmalloc should segregate medium-sized objects by line like it does for small-sized objects https://bugs.webkit.org/show_bug.cgi?id=136693 Reviewed by Gavin Barraclough. 4% reduction in heap size on the MallocBench *_memory_warning benchmarks. No throughput change. We keep an array of medium allocators, just like our array of small allocators. In future, we can simplify the allocation fast path by merging the small and medium allocator arrays. For now, this is the simplest change that gets the win. * bmalloc/Allocator.cpp: (bmalloc::Allocator::Allocator): (bmalloc::Allocator::scavenge): (bmalloc::Allocator::allocateMedium): * bmalloc/Allocator.h: * bmalloc/Sizes.h: (bmalloc::Sizes::mediumSizeClassFor): 2014-09-11 Geoffrey Garen Reviewed by Sam Weinig. Renamed log => retire for clarity. * bmalloc/Allocator.cpp: (bmalloc::Allocator::scavenge): (bmalloc::Allocator::retire): (bmalloc::Allocator::allocateMedium): (bmalloc::Allocator::allocateSlowCase): (bmalloc::Allocator::log): Deleted. * bmalloc/Allocator.h: 2014-09-11 Geoffrey Garen bmalloc: eager scavenge leaves behind a bogus allocator https://bugs.webkit.org/show_bug.cgi?id=136743 Reviewed by Sam Weinig. Be sure to clear the allocator after logging it in the eager scavenge case, so that we don't later try to allocate out of the lines that we have thrown away. We didn't need to do this previously because scavenge would only happen at thread exit time, after which no further allocation from the per-thread cache would take place. * bmalloc/Allocator.cpp: (bmalloc::Allocator::scavenge): * bmalloc/MediumAllocator.h: (bmalloc::MediumAllocator::clear): * bmalloc/SmallAllocator.h: (bmalloc::SmallAllocator::clear): 2014-09-05 Geoffrey Garen bmalloc should honor the FastMalloc statistics API https://bugs.webkit.org/show_bug.cgi?id=136592 Reviewed by Gavin Barraclough. We do this by tracking "size" and "capacity" in the VM heap. The VM heap's "capacity" is all the VM we ever allocated. The VM heap's "size" the subset of VM currently held onto by the VM heap (and therefore not in use by the regular heap). Somewhat ironically, reducing the process's memory footprint, increases the size of the VM heap, since the VM heap holds the pages that are purely virtual and not physical. * bmalloc/Heap.cpp: (bmalloc::Heap::size): (bmalloc::Heap::capacity): * bmalloc/Heap.h: * bmalloc/VMHeap.cpp: (bmalloc::VMHeap::VMHeap): (bmalloc::VMHeap::allocateSmallChunk): (bmalloc::VMHeap::allocateMediumChunk): (bmalloc::VMHeap::allocateLargeChunk): * bmalloc/VMHeap.h: (bmalloc::VMHeap::size): (bmalloc::VMHeap::capacity): (bmalloc::VMHeap::allocateSmallPage): (bmalloc::VMHeap::allocateMediumPage): (bmalloc::VMHeap::allocateLargeRange): (bmalloc::VMHeap::deallocateSmallPage): (bmalloc::VMHeap::deallocateMediumPage): (bmalloc::VMHeap::deallocateLargeRange): * bmalloc/bmalloc.h: (bmalloc::api::heapSize): (bmalloc::api::heapCapacity): 2014-09-02 Geoffrey Garen bmalloc crashes on the EWS bots (due to bad large object allocation) https://bugs.webkit.org/show_bug.cgi?id=136469 Reviewed by Andreas Kling. It's possible to convince bmalloc to perform a bad large object allocation, through these steps: (1) Insert object A into freelist F0. (2) Split, merge and split again A's neighbors such that object B is inserted into freelist F0, with boundary tag and size equal to object A, but pointer not completely equal to object A. Put object B at the head of F0. (3) Allocate some other object from F0, swapping its position in the freelist with object B, such that object A is now ahead of object B. --> Now, the next allocation for size A/B will allocate object A, which has a slightly wrong idea about where the object actually begins. Immediately, you'll corrupt a little memory, and over time, you'll also corrupt boundary tag metadata. The solution is to store the begin pointer in the boundary tag. Luckily, this doesn't make the tag any bigger, and it's not a noticeable slowdown on MallocBench. * bmalloc/Algorithm.h: (bmalloc::rightShift): * bmalloc/BeginTag.h: (bmalloc::BeginTag::isInFreeList): This is the bug fix. Make sure to validate the start pointer when popping off the free list. Through a very uncommon set of steps, it is possible to have an item in the free list that is valid by all accounts except for its start pointer. * bmalloc/BoundaryTag.h: (bmalloc::BoundaryTag::compactBegin): (bmalloc::BoundaryTag::setRange): (bmalloc::BoundaryTag::setSize): Deleted. Record a compact version of the start pointer. We don't need the whole pointer -- just the offset, in largeAlignment increments, into the relevant boundary tag bucket. * bmalloc/BoundaryTagInlines.h: (bmalloc::validateNext): (bmalloc::BoundaryTag::init): (bmalloc::BoundaryTag::mergeLarge): (bmalloc::BoundaryTag::splitLarge): * bmalloc/SegregatedFreeList.cpp: (bmalloc::SegregatedFreeList::insert): (bmalloc::SegregatedFreeList::takeGreedy): (bmalloc::SegregatedFreeList::take): Provide the whole range instead of the size when establishing a boundary tag, as required by the new interface. * bmalloc/Sizes.h: 2014-08-14 Geoffrey Garen Fixed a bmalloc crash seen on the EWS bot https://bugs.webkit.org/show_bug.cgi?id=135955 Reviewed by Andreas Kling. * bmalloc/Syscall.h: Some CG APIs vm_copy their input buffers. If the input buffer is a malloc region, that region will get marked Copy-On-Write by the kernel. Calls to madvise() for COW regions fail and return EINVAL on older OS X's. In 10.10, they still fail, but they do not return EINVAL. So, we can only ASSERT that our syscalls succeed starting with 10.10. 2014-08-14 Geoffrey Garen Fixed the bmalloc build https://bugs.webkit.org/show_bug.cgi?id=135953 Reviewed by Andreas Kling. * bmalloc.xcodeproj/project.pbxproj: Marked a few headers as private. These headers are used, so they must be available outside the project. 2014-08-13 Daniel Bates Attempt to fix the build following (https://bugs.webkit.org/show_bug.cgi?id=135895) Substitute PerThreadStorage::initSharedKeyIfNeeded() for initSharedKeyIfNeeded() in implementation of PerThread::getFastCase(). * bmalloc/PerThread.h: (bmalloc::PerThread::getFastCase): 2014-08-13 Daniel Bates Make bmalloc::PerThread work without C++ thread local storage https://bugs.webkit.org/show_bug.cgi?id=135895 Reviewed by Geoffrey Garen. Implement support for building bmalloc without C++ thread local storage. * bmalloc/BPlatform.h: Remove macro define BPLATFORM_IOS_SIMULATOR. Added macro function BCOMPILER_SUPPORTS() and macro define BCOMPILER_SUPPORTS_CXX_THREAD_LOCAL that can be used to determine whether the compiler supports C++ thread local storage. * bmalloc/PerThread.h: (bmalloc::PerThreadStorage::get): Modified to call pthread_getspecific() when building without C++ thread local storage. (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Added. (bmalloc::PerThreadStorage::init): Moved logic to initialize shared Pthread key from here to PerThreadStorage::initSharedKeyIfNeeded(). (bmalloc::PerThread::getFastCase): Modified to call PerThreadStorage::initSharedKeyIfNeeded() before querying PerThreadStorage::get() when building without C++ thread local storage so as to ensure that the shared key has been initialized. (_pthread_setspecific_direct): Deleted. (_pthread_getspecific_direct): Deleted. 2014-08-13 Daniel Bates [iOS] Make JavaScriptCore and bmalloc build with the public SDK https://bugs.webkit.org/show_bug.cgi?id=135848 Reviewed by Geoffrey Garen. * bmalloc/BPlatform.h: Added macro BPLATFORM_IOS_SIMULATOR, which evaluates to true when building for the iOS Simulator. * bmalloc/PerThread.h: Use pthread_machdep.h code path when building for iOS Simulator using the public SDK. (_pthread_setspecific_direct): Added; only defined when building for the iOS Simulator using the public SDK. (_pthread_getspecific_direct): Added; only defined when building for the iOS Simulator using the public SDK. 2014-08-12 Daniel Bates BPLATFORM(IOS) always evaluates to false https://bugs.webkit.org/show_bug.cgi?id=135843 Reviewed by Geoffrey Garen. Fix typo in definition of BPLATFORM() and include system header TargetConditionals.h (when building on an Apple platform) so that BPLATFORM(X) evaluates to true when building for platform X. In particular, so that BPLATFORM(IOS) evaluates to true when building for iOS. As a side effect of this change, the change made in will be honored and iOS will assume a VM page size of 16kB (again) instead of 4kB. * bmalloc/BPlatform.h: 2014-08-11 Andy Estes [iOS] Get rid of iOS.xcconfig https://bugs.webkit.org/show_bug.cgi?id=135809 Reviewed by Joseph Pecoraro. All iOS.xcconfig did was include AspenFamily.xcconfig, so there's no need for the indirection. * Configurations/Base.xcconfig: * Configurations/iOS.xcconfig: Removed. * bmalloc.xcodeproj/project.pbxproj: 2014-05-01 Dan Bernstein Fixed production builds for the iOS Simulator. * Configurations/bmalloc.xcconfig: Include INSTALL_PATH_PREFIX in PRIVATE_HEADERS_FOLDER_PATH when installing. 2014-04-20 Geoffrey Garen bmalloc: Segregate pages by objects size https://bugs.webkit.org/show_bug.cgi?id=131909 Reviewed by Andreas Kling. 2% reduction in memory-at-end on the Membuster memory_warning benchmarks. * bmalloc/Allocator.cpp: (bmalloc::Allocator::allocateSlowCase): * bmalloc/Allocator.h: (bmalloc::Allocator::allocateFastCase): (bmalloc::Allocator::smallAllocatorFor): Use the new shared helper function for size class calculation. * bmalloc/Deallocator.cpp: (bmalloc::Deallocator::Deallocator): (bmalloc::Deallocator::scavenge): (bmalloc::Deallocator::deallocateSmallLine): (bmalloc::Deallocator::allocateSmallLine): * bmalloc/Deallocator.h: Keep a cache for every size class, since the cache can't be shared anymore. * bmalloc/Heap.cpp: (bmalloc::Heap::allocateSmallLineSlowCase): * bmalloc/Heap.h: (bmalloc::Heap::deallocateSmallLine): Ditto. (bmalloc::Heap::allocateSmallLine): Check size class in addition to page refcount when allocating a line because we might have deallocated the page and the recycled it for another size class. (bmalloc::Heap::deallocateMediumLine): (bmalloc::Heap::allocateMediumLine): * bmalloc/Line.h: (bmalloc::Line::refCount): * bmalloc/Page.h: (bmalloc::Page::refCount): (bmalloc::Page::smallSizeClass): (bmalloc::Page::setSmallSizeClass): (bmalloc::Page::refCount): Deleted. * bmalloc/Sizes.h: (bmalloc::Sizes::smallSizeClassFor): New shared API for computing an index into an array from a size. 2014-04-19 Geoffrey Garen bmalloc: Improved alignment in LargeChunk https://bugs.webkit.org/show_bug.cgi?id=131895 Reviewed by Andreas Kling. * bmalloc/Chunk.h: * bmalloc/LargeChunk.h: Align to vmPageSize just like Chunk does. Technically, the previous alignment was harmless, but I would prefer, dear reader, not to have to explain the interlocking set of circumstances that made it so. 2014-04-19 Geoffrey Garen Rolled out r167502 because it caused a crash on the facebook benchmark. Unreviewed. bmalloc: Added an XSmall line size https://bugs.webkit.org/show_bug.cgi?id=131851 Reviewed by Sam Weinig. 2014-04-19 Geoffrey Garen bmalloc: Mutex should be harder to use wrong https://bugs.webkit.org/show_bug.cgi?id=131879 Reviewed by Andreas Kling. Mutex now has a proper constructor, so you can't deadlock by forgetting to initialize it. * bmalloc.xcodeproj/project.pbxproj: * bmalloc/Allocator.cpp: (bmalloc::Allocator::processXSmallAllocatorLog): (bmalloc::Allocator::processSmallAllocatorLog): (bmalloc::Allocator::processMediumAllocatorLog): (bmalloc::Allocator::allocateLarge): (bmalloc::Allocator::allocateXLarge): Global replace Mutex => StaticMutex, since the Heap mutex is a static. * bmalloc/AsyncTask.h: (bmalloc::Function>::AsyncTask): Use Mutex, since we're not static. No need for explicit initialization anymore. * bmalloc/Deallocator.cpp: (bmalloc::Deallocator::scavenge): (bmalloc::Deallocator::deallocateLarge): (bmalloc::Deallocator::deallocateXLarge): (bmalloc::Deallocator::processObjectLog): (bmalloc::Deallocator::deallocateSmallLine): (bmalloc::Deallocator::deallocateXSmallLine): (bmalloc::Deallocator::allocateSmallLine): (bmalloc::Deallocator::allocateXSmallLine): (bmalloc::Deallocator::deallocateMediumLine): (bmalloc::Deallocator::allocateMediumLine): * bmalloc/Deallocator.h: * bmalloc/Heap.cpp: (bmalloc::sleep): (bmalloc::Heap::Heap): (bmalloc::Heap::concurrentScavenge): (bmalloc::Heap::scavenge): (bmalloc::Heap::scavengeSmallPages): (bmalloc::Heap::scavengeXSmallPages): (bmalloc::Heap::scavengeMediumPages): (bmalloc::Heap::scavengeLargeRanges): (bmalloc::Heap::allocateXSmallLineSlowCase): (bmalloc::Heap::allocateSmallLineSlowCase): (bmalloc::Heap::allocateMediumLineSlowCase): (bmalloc::Heap::allocateXLarge): (bmalloc::Heap::deallocateXLarge): (bmalloc::Heap::allocateLarge): (bmalloc::Heap::deallocateLarge): * bmalloc/Heap.h: (bmalloc::Heap::deallocateXSmallLine): (bmalloc::Heap::allocateXSmallLine): (bmalloc::Heap::deallocateSmallLine): (bmalloc::Heap::allocateSmallLine): (bmalloc::Heap::deallocateMediumLine): (bmalloc::Heap::allocateMediumLine): * bmalloc/Line.h: (bmalloc::Line::deref): * bmalloc/Mutex.cpp: Removed. * bmalloc/Mutex.h: (bmalloc::Mutex::Mutex): (bmalloc::Mutex::init): Deleted. (bmalloc::Mutex::try_lock): Deleted. (bmalloc::Mutex::lock): Deleted. (bmalloc::Mutex::unlock): Deleted. * bmalloc/Page.h: (bmalloc::Page::ref): (bmalloc::Page::deref): (bmalloc::Page::refCount): * bmalloc/PerProcess.h: (bmalloc::PerProcess::mutex): (bmalloc::PerProcess::getSlowCase): * bmalloc/StaticMutex.cpp: Added. (bmalloc::StaticMutex::lockSlowCase): * bmalloc/StaticMutex.h: Added. (bmalloc::StaticMutex::init): (bmalloc::StaticMutex::try_lock): (bmalloc::StaticMutex::lock): (bmalloc::StaticMutex::unlock): * bmalloc/VMHeap.h: (bmalloc::VMHeap::deallocateXSmallPage): (bmalloc::VMHeap::deallocateSmallPage): (bmalloc::VMHeap::deallocateMediumPage): (bmalloc::VMHeap::deallocateLargeRange): * bmalloc/bmalloc.h: (bmalloc::api::scavenge): Global replace Mutex => StaticMutex, since the Heap mutex is a static. 2014-04-18 Geoffrey Garen bmalloc: AsyncTask should use Mutex instead of std::mutex https://bugs.webkit.org/show_bug.cgi?id=131865 Reviewed by Gavin Barraclough. std::mutex is so slow that it makes parallelizing simple tasks through AsyncTask a net regression. Mutex fixes this. * bmalloc/AsyncTask.h: (bmalloc::Function>::AsyncTask): (bmalloc::Function>::join): (bmalloc::Function>::runSlowCase): (bmalloc::Function>::entryPoint): * bmalloc/Mutex.h: (bmalloc::Mutex::init): 2014-04-18 Geoffrey Garen bmalloc: Added an XSmall line size https://bugs.webkit.org/show_bug.cgi?id=131851 Reviewed by Sam Weinig. Reduces malloc footprint on Membuster recordings by 10%. This is a throughput regression, but we're still way ahead of TCMalloc. I have some ideas for how to recover the regression -- but I wanted to get this win in first. Full set of benchmark results: bmalloc> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks --measure-heap nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/ nopatch patch Δ Peak Memory: reddit_memory_warning 7,896kB 7,532kB ^ 1.05x smaller flickr_memory_warning 12,968kB 12,324kB ^ 1.05x smaller theverge_memory_warning 16,672kB 15,200kB ^ 1.1x smaller 11,952kB 11,216kB ^ 1.07x smaller 12,512kB 11,685kB ^ 1.07x smaller 11,375kB 10,726kB ^ 1.06x smaller Memory at End: reddit_memory_warning 7,320kB 6,856kB ^ 1.07x smaller flickr_memory_warning 10,848kB 9,692kB ^ 1.12x smaller theverge_memory_warning 16,380kB 14,872kB ^ 1.1x smaller 10,916kB 9,961kB ^ 1.1x smaller 11,516kB 10,473kB ^ 1.1x smaller 10,350kB 9,485kB ^ 1.09x smaller MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/ nopatch patch Δ Execution Time: churn 127ms 151ms ! 1.19x slower list_allocate 130ms 164ms ! 1.26x slower tree_allocate 109ms 127ms ! 1.17x slower tree_churn 115ms 120ms ! 1.04x slower facebook 240ms 259ms ! 1.08x slower fragment 91ms 131ms ! 1.44x slower fragment_iterate 105ms 106ms ! 1.01x slower message_one 260ms 259ms ^ 1.0x faster message_many 149ms 154ms ! 1.03x slower medium 194ms 248ms ! 1.28x slower big 157ms 160ms ! 1.02x slower 144ms 163ms ! 1.13x slower 152ms 171ms ! 1.12x slower 137ms 156ms ! 1.14x slower MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/ nopatch patch Δ Execution Time: churn 126ms 148ms ! 1.17x slower churn --parallel 62ms 76ms ! 1.23x slower list_allocate 130ms 164ms ! 1.26x slower list_allocate --parallel 120ms 175ms ! 1.46x slower tree_allocate 111ms 127ms ! 1.14x slower tree_allocate --parallel 95ms 135ms ! 1.42x slower tree_churn 115ms 124ms ! 1.08x slower tree_churn --parallel 107ms 126ms ! 1.18x slower facebook 240ms 276ms ! 1.15x slower facebook --parallel 802ms 1,088ms ! 1.36x slower fragment 92ms 130ms ! 1.41x slower fragment --parallel 66ms 124ms ! 1.88x slower fragment_iterate 109ms 127ms ! 1.17x slower fragment_iterate --parallel 55ms 64ms ! 1.16x slower message_one 260ms 260ms message_many 170ms 238ms ! 1.4x slower medium 185ms 250ms ! 1.35x slower medium --parallel 210ms 334ms ! 1.59x slower big 150ms 169ms ! 1.13x slower big --parallel 138ms 144ms ! 1.04x slower 135ms 170ms ! 1.26x slower 167ms 214ms ! 1.28x slower 117ms 148ms ! 1.26x slower MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/ TC patch Δ Peak Memory: reddit_memory_warning 13,836kB 13,436kB ^ 1.03x smaller flickr_memory_warning 24,868kB 25,188kB ! 1.01x bigger theverge_memory_warning 24,504kB 26,636kB ! 1.09x bigger 20,353kB 20,812kB ! 1.02x bigger 21,069kB 21,753kB ! 1.03x bigger 19,570kB 19,780kB ! 1.01x bigger Memory at End: reddit_memory_warning 8,656kB 10,016kB ! 1.16x bigger flickr_memory_warning 11,844kB 13,784kB ! 1.16x bigger theverge_memory_warning 18,516kB 22,748kB ! 1.23x bigger 12,382kB 14,644kB ! 1.18x bigger 13,005kB 15,516kB ! 1.19x bigger 11,813kB 13,867kB ! 1.17x bigger MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/ TC patch Δ Execution Time: churn 416ms 148ms ^ 2.81x faster list_allocate 463ms 164ms ^ 2.82x faster tree_allocate 292ms 127ms ^ 2.3x faster tree_churn 157ms 120ms ^ 1.31x faster facebook 327ms 276ms ^ 1.18x faster fragment 335ms 129ms ^ 2.6x faster fragment_iterate 344ms 108ms ^ 3.19x faster message_one 386ms 258ms ^ 1.5x faster message_many 410ms 154ms ^ 2.66x faster medium 391ms 245ms ^ 1.6x faster big 261ms 167ms ^ 1.56x faster 332ms 164ms ^ 2.02x faster 344ms 172ms ^ 1.99x faster 317ms 157ms ^ 2.02x faster * bmalloc.xcodeproj/project.pbxproj: * bmalloc/Allocator.cpp: (bmalloc::Allocator::Allocator): Don't assume that each allocator's index corresponds with its size. Instead, use the size selection function explicitly. Now that we have XSmall, some small allocator entries are unused. (bmalloc::Allocator::scavenge): (bmalloc::Allocator::log): (bmalloc::Allocator::processXSmallAllocatorLog): (bmalloc::Allocator::allocateSlowCase): * bmalloc/Allocator.h: (bmalloc::Allocator::xSmallAllocatorFor): (bmalloc::Allocator::allocateFastCase): * bmalloc/Chunk.h: * bmalloc/Deallocator.cpp: (bmalloc::Deallocator::scavenge): (bmalloc::Deallocator::processObjectLog): (bmalloc::Deallocator::deallocateSlowCase): (bmalloc::Deallocator::deallocateXSmallLine): (bmalloc::Deallocator::allocateXSmallLine): * bmalloc/Deallocator.h: (bmalloc::Deallocator::deallocateFastCase): * bmalloc/Heap.cpp: (bmalloc::Heap::scavenge): (bmalloc::Heap::scavengeXSmallPages): (bmalloc::Heap::allocateXSmallLineSlowCase): * bmalloc/Heap.h: (bmalloc::Heap::deallocateXSmallLine): (bmalloc::Heap::allocateXSmallLine): * bmalloc/LargeChunk.h: (bmalloc::LargeChunk::get): (bmalloc::LargeChunk::endTag): * bmalloc/Line.h: * bmalloc/MediumAllocator.h: (bmalloc::MediumAllocator::allocate): (bmalloc::MediumAllocator::refill): * bmalloc/ObjectType.cpp: (bmalloc::objectType): * bmalloc/ObjectType.h: (bmalloc::isXSmall): (bmalloc::isSmall): (bmalloc::isMedium): (bmalloc::isLarge): (bmalloc::isSmallOrMedium): Deleted. * bmalloc/SegregatedFreeList.h: I boiler-plate copied existing code for handling small objects. There's probably a reasonable way to share this code in the future -- I'll look into that once it's stopped changing. * bmalloc/Sizes.h: Tweaked size classes to make Membuster happy. This is the main reason things got slower. * bmalloc/SmallAllocator.h: (bmalloc::SmallAllocator::allocate): * bmalloc/SmallTraits.h: * bmalloc/VMHeap.cpp: (bmalloc::VMHeap::allocateXSmallChunk): * bmalloc/VMHeap.h: (bmalloc::VMHeap::allocateXSmallPage): (bmalloc::VMHeap::deallocateXSmallPage): * bmalloc/XSmallAllocator.h: Added. (bmalloc::XSmallAllocator::isNull): (bmalloc::XSmallAllocator::canAllocate): (bmalloc::XSmallAllocator::XSmallAllocator): (bmalloc::XSmallAllocator::line): (bmalloc::XSmallAllocator::allocate): (bmalloc::XSmallAllocator::objectCount): (bmalloc::XSmallAllocator::derefCount): (bmalloc::XSmallAllocator::refill): * bmalloc/XSmallChunk.h: Added. * bmalloc/XSmallLine.h: Added. * bmalloc/XSmallPage.h: Added. * bmalloc/XSmallTraits.h: Added. * bmalloc/bmalloc.h: (bmalloc::api::realloc): Boiler-plate copy, as above. 2014-04-14 Geoffrey Garen MallocBench should scavenge explicitly instead of waiting https://bugs.webkit.org/show_bug.cgi?id=131661 Reviewed by Andreas Kling. Added explicit scavenge support to bmalloc. This isn't a memory win, since bmalloc's per-thread cache is so small. But it makes testing simpler. * bmalloc/Allocator.cpp: (bmalloc::Allocator::~Allocator): (bmalloc::Allocator::scavenge): * bmalloc/Allocator.h: * bmalloc/Cache.cpp: (bmalloc::Cache::operator new): (bmalloc::Cache::operator delete): (bmalloc::Cache::Cache): (bmalloc::Cache::scavenge): * bmalloc/Cache.h: * bmalloc/Deallocator.cpp: (bmalloc::Deallocator::~Deallocator): (bmalloc::Deallocator::scavenge): * bmalloc/Deallocator.h: Factored existing scavenging code into helper functions, for reuse. * bmalloc/Heap.cpp: (bmalloc::sleep): (bmalloc::Heap::concurrentScavenge): (bmalloc::Heap::scavenge): (bmalloc::Heap::scavengeSmallPages): (bmalloc::Heap::scavengeMediumPages): (bmalloc::Heap::scavengeLargeRanges): * bmalloc/Heap.h: Made scavenge sleep duration a parameter. Forced scavenging -- in response to a benchmark or a low memory warning -- wants to complete as soon as possible, so its sleep duration is 0. * bmalloc/bmalloc.h: (bmalloc::api::scavenge): * bmalloc/mbmalloc.cpp: Exported the scavenge API for MallocBench's use. 2014-04-14 Geoffrey Garen Use 4kB pages on Mac https://bugs.webkit.org/show_bug.cgi?id=131658 Reviewed by Sam Weinig. This reduces memory use a lot on Membuster: base patch Δ Execution Time: reddit_memory_warning 18ms 17ms ^ 1.06x faster flickr_memory_warning 34ms 36ms ! 1.06x slower theverge_memory_warning 39ms 41ms ! 1.05x slower 29ms 29ms ! 1.02x slower 30ms 31ms ! 1.03x slower 27ms 27ms ^ 1.0x faster Peak Memory: reddit_memory_warning 16,412kB 16,436kB ! 1.0x bigger flickr_memory_warning 30,120kB 30,184kB ! 1.0x bigger theverge_memory_warning 33,408kB 33,420kB ! 1.0x bigger 25,466kB 25,499kB ! 1.0x bigger 26,647kB 26,680kB ! 1.0x bigger 24,181kB 24,214kB ! 1.0x bigger Memory at End: reddit_memory_warning 2,404kB 1,920kB ^ 1.25x smaller flickr_memory_warning 3,764kB 3,072kB ^ 1.23x smaller theverge_memory_warning 3,648kB 3,132kB ^ 1.16x smaller 3,208kB 2,644kB ^ 1.21x smaller 3,272kB 2,708kB ^ 1.21x smaller 3,139kB 2,574kB ^ 1.22x smaller * bmalloc.xcodeproj/project.pbxproj: * bmalloc/BPlatform.h: Added. * bmalloc/VMAllocate.h: Only use 16kB pages on iOS because the page size is 4kB on Mac. 2014-04-14 Alexey Proskuryakov Fixed svn:ignore on bmalloc.xcodeproj, it had erroneous leading spaces. * bmalloc.xcodeproj: Modified property svn:ignore. 2014-04-13 Geoffrey Garen Fixed some mbmalloc exports https://bugs.webkit.org/show_bug.cgi?id=131599 Reviewed by Ryosuke Niwa. * bmalloc.xcodeproj/project.pbxproj: Made some headers a private part of the project, so we can call them from API. * bmalloc/mbmalloc.cpp: Marked the mbmalloc functions with default visibility, so they show up as exported in the .dylib. 2014-04-09 Geoffrey Garen Put bmalloc headers in the right place https://bugs.webkit.org/show_bug.cgi?id=131464 Reviewed by Mark Rowe. * Configurations/bmalloc.xcconfig: Set PRIVATE_HEADERS_FOLDER_PATH to specify that we don't just want to dump all of our generically-named headers into /usr/local/include. 2014-04-08 Geoffrey Garen Made bmalloc more #include friendly https://bugs.webkit.org/show_bug.cgi?id=131386 Reviewed by Andreas Kling. Marked a bunch of headers private so they can be used from client code that #includes bmalloc.h. Renamed ASSERT macros to BASSERT. This matches their header, which already had to be renamed, and fixes conflicts with WTF's ASSERT macros. * bmalloc.xcodeproj/project.pbxproj: * bmalloc/Allocator.cpp: (bmalloc::Allocator::allocateSlowCase): * bmalloc/AsyncTask.h: (bmalloc::Function>::runSlowCase): * bmalloc/BAssert.h: * bmalloc/BoundaryTag.h: (bmalloc::BoundaryTag::setSize): * bmalloc/BoundaryTagInlines.h: (bmalloc::validate): (bmalloc::BoundaryTag::init): (bmalloc::BoundaryTag::deallocate): (bmalloc::BoundaryTag::splitLarge): (bmalloc::BoundaryTag::allocate): * bmalloc/Chunk.h: * bmalloc/Deallocator.cpp: (bmalloc::Deallocator::processObjectLog): (bmalloc::Deallocator::deallocateSlowCase): * bmalloc/Deallocator.h: (bmalloc::Deallocator::deallocateFastCase): * bmalloc/FixedVector.h: (bmalloc::Capacity>::operator): (bmalloc::Capacity>::push): (bmalloc::Capacity>::pop): (bmalloc::Capacity>::shrink): * bmalloc/Heap.cpp: (bmalloc::Heap::allocateLarge): * bmalloc/LargeChunk.h: (bmalloc::LargeChunk::get): (bmalloc::LargeChunk::endTag): * bmalloc/Line.h: (bmalloc::Line::concurrentRef): (bmalloc::Line::deref): * bmalloc/MediumAllocator.h: (bmalloc::MediumAllocator::allocate): * bmalloc/ObjectType.h: (bmalloc::isSmall): * bmalloc/Page.h: (bmalloc::Page::ref): (bmalloc::Page::deref): * bmalloc/PerThread.h: (bmalloc::PerThread::getSlowCase): * bmalloc/SegregatedFreeList.cpp: (bmalloc::SegregatedFreeList::SegregatedFreeList): (bmalloc::SegregatedFreeList::insert): * bmalloc/SmallAllocator.h: (bmalloc::SmallAllocator::allocate): (bmalloc::SmallAllocator::refill): * bmalloc/Syscall.h: * bmalloc/VMAllocate.h: (bmalloc::vmValidate): (bmalloc::vmAllocate): (bmalloc::vmDeallocatePhysicalPagesSloppy): * bmalloc/Vector.h: (bmalloc::Vector::operator): (bmalloc::Vector::pop): (bmalloc::Vector::shrink): * bmalloc/XLargeChunk.h: (bmalloc::XLargeChunk::range): (bmalloc::XLargeChunk::size): 2014-04-08 Geoffrey Garen Removed an unused file. Unreviewed. * bmalloc/AsyncTask.cpp: Removed. 2014-04-07 Geoffrey Garen Build bmalloc on Mac https://bugs.webkit.org/show_bug.cgi?id=131333 Reviewed by Mark Rowe. * Makefile: Added. For make clients. These files are required for building any project in WebKit. I copied them from WTF: * Configurations: Added. * Configurations/Base.xcconfig: Added. * Configurations/DebugRelease.xcconfig: Added. * Configurations/bmalloc.xcconfig: Added. * Configurations/iOS.xcconfig: Added. * Configurations/mbmalloc.xcconfig: Added. * bmalloc.xcodeproj/project.pbxproj: I removed per-project-file stuff from here because everything is in .xcconfig files now. I had to fix a bunch of minor warnings, since they're enabled in our .xcconfig files: * bmalloc/AsyncTask.h: (bmalloc::Function>::AsyncTask): * bmalloc/BAssert.h: * bmalloc/BoundaryTagInlines.h: (bmalloc::validate): * bmalloc/Heap.cpp: (bmalloc::Heap::Heap): (bmalloc::Heap::allocateLarge): (bmalloc::Heap::deallocateLarge): * bmalloc/Mutex.h: (bmalloc::Mutex::Mutex): Deleted. * bmalloc/VMAllocate.h: (bmalloc::vmValidate): * bmalloc/mbmalloc.cpp: 2014-04-07 Geoffrey Garen bmalloc: Fixed a leak in the per-thread cache https://bugs.webkit.org/show_bug.cgi?id=131330 Reviewed by Andreas Kling. Remember to deallocate our line caches upon thread exit. * bmalloc/Deallocator.cpp: (bmalloc::Deallocator::~Deallocator): 2014-04-07 Geoffrey Garen bmalloc: rolled out the tryLock experiment https://bugs.webkit.org/show_bug.cgi?id=131328 Reviewed by Andreas Kling. It wasn't a speedup. * bmalloc.xcodeproj/project.pbxproj: * bmalloc/Allocator.cpp: (bmalloc::Allocator::processSmallAllocatorLog): (bmalloc::Allocator::processMediumAllocatorLog): * bmalloc/Deallocator.cpp: (bmalloc::Deallocator::processObjectLog): (bmalloc::Deallocator::deallocateSlowCase): (bmalloc::Deallocator::deallocateSmallLine): (bmalloc::Deallocator::deallocateMediumLine): * bmalloc/Deallocator.h: (bmalloc::Deallocator::deallocateFastCase): * bmalloc/Heap.h: (bmalloc::Heap::deallocateSmallLine): (bmalloc::Heap::deallocateMediumLine): * bmalloc/Line.h: (bmalloc::Line::deref): * bmalloc/Page.h: (bmalloc::Page::deref): 2014-04-07 Geoffrey Garen bmalloc https://bugs.webkit.org/show_bug.cgi?id=131170 Reviewed by Andreas Kling. Initial commit. * bmalloc: Added. * bmalloc.xcodeproj: Added. * bmalloc.xcodeproj/project.pbxproj: Added. * bmalloc/Algorithm.h: Added. (bmalloc::max): (bmalloc::min): (bmalloc::mask): (bmalloc::test): (bmalloc::roundUpToMultipleOf): (bmalloc::roundDownToMultipleOf): (bmalloc::sizeOf): (bmalloc::bitCount): (bmalloc::isPowerOfTwo): * bmalloc/Allocator.cpp: Added. (bmalloc::Allocator::Allocator): (bmalloc::Allocator::~Allocator): (bmalloc::Allocator::log): (bmalloc::Allocator::processSmallAllocatorLog): (bmalloc::Allocator::processMediumAllocatorLog): (bmalloc::Allocator::allocateLarge): (bmalloc::Allocator::allocateXLarge): (bmalloc::Allocator::allocateMedium): (bmalloc::Allocator::allocateSlowCase): * bmalloc/Allocator.h: Added. (bmalloc::Allocator::smallAllocatorFor): (bmalloc::Allocator::allocateFastCase): (bmalloc::Allocator::allocate): * bmalloc/AsyncTask.cpp: Added. (bmalloc::AsyncTask::runSlowCase): (bmalloc::AsyncTask::pthreadEntryPoint): (bmalloc::AsyncTask::entryPoint): * bmalloc/AsyncTask.h: Added. (bmalloc::Function>::AsyncTask): (bmalloc::Function>::join): (bmalloc::Function>::run): (bmalloc::Function>::runSlowCase): (bmalloc::Function>::pthreadEntryPoint): (bmalloc::Function>::entryPoint): * bmalloc/BAssert.h: Added. * bmalloc/BeginTag.h: Added. (bmalloc::BeginTag::isInFreeList): * bmalloc/BoundaryTag.h: Added. (bmalloc::BoundaryTag::isXLarge): (bmalloc::BoundaryTag::setXLarge): (bmalloc::BoundaryTag::isFree): (bmalloc::BoundaryTag::setFree): (bmalloc::BoundaryTag::isEnd): (bmalloc::BoundaryTag::setEnd): (bmalloc::BoundaryTag::hasPhysicalPages): (bmalloc::BoundaryTag::setHasPhysicalPages): (bmalloc::BoundaryTag::isNull): (bmalloc::BoundaryTag::clear): (bmalloc::BoundaryTag::size): (bmalloc::BoundaryTag::setSize): (bmalloc::BoundaryTag::prev): (bmalloc::BoundaryTag::next): * bmalloc/BoundaryTagInlines.h: Added. (bmalloc::validate): (bmalloc::validatePrev): (bmalloc::validateNext): (bmalloc::BoundaryTag::init): (bmalloc::BoundaryTag::mergeLargeLeft): (bmalloc::BoundaryTag::mergeLargeRight): (bmalloc::BoundaryTag::mergeLarge): (bmalloc::BoundaryTag::deallocate): (bmalloc::BoundaryTag::splitLarge): (bmalloc::BoundaryTag::allocate): * bmalloc/Cache.cpp: Added. (bmalloc::Cache::operator new): (bmalloc::Cache::operator delete): (bmalloc::Cache::Cache): (bmalloc::Cache::allocateSlowCase): (bmalloc::Cache::allocateSlowCaseNullCache): (bmalloc::Cache::deallocateSlowCase): (bmalloc::Cache::deallocateSlowCaseNullCache): * bmalloc/Cache.h: Added. (bmalloc::Cache::allocator): (bmalloc::Cache::deallocator): (bmalloc::Cache::allocateFastCase): (bmalloc::Cache::deallocateFastCase): (bmalloc::Cache::allocate): (bmalloc::Cache::deallocate): * bmalloc/Chunk.h: Added. (bmalloc::Chunk::begin): (bmalloc::Chunk::end): (bmalloc::Chunk::lines): (bmalloc::Chunk::pages): * bmalloc/Deallocator.cpp: Added. (bmalloc::Deallocator::Deallocator): (bmalloc::Deallocator::~Deallocator): (bmalloc::Deallocator::deallocateLarge): (bmalloc::Deallocator::deallocateXLarge): (bmalloc::Deallocator::processObjectLog): (bmalloc::Deallocator::deallocateSlowCase): (bmalloc::Deallocator::deallocateSmallLine): (bmalloc::Deallocator::allocateSmallLine): (bmalloc::Deallocator::deallocateMediumLine): (bmalloc::Deallocator::allocateMediumLine): * bmalloc/Deallocator.h: Added. (bmalloc::Deallocator::deallocateFastCase): (bmalloc::Deallocator::deallocate): * bmalloc/EndTag.h: Added. (bmalloc::EndTag::operator=): * bmalloc/FixedVector.h: Added. (bmalloc::FixedVector::begin): (bmalloc::FixedVector::end): (bmalloc::FixedVector::size): (bmalloc::FixedVector::capacity): (bmalloc::FixedVector::clear): (bmalloc::FixedVector::isEmpty): (bmalloc::Capacity>::FixedVector): (bmalloc::Capacity>::operator): (bmalloc::Capacity>::push): (bmalloc::Capacity>::pop): (bmalloc::Capacity>::shrink): * bmalloc/Heap.cpp: Added. (bmalloc::sleep): (bmalloc::Heap::Heap): (bmalloc::Heap::concurrentScavenge): (bmalloc::Heap::scavengeSmallPages): (bmalloc::Heap::scavengeMediumPages): (bmalloc::Heap::scavengeLargeRanges): (bmalloc::Heap::allocateSmallLineSlowCase): (bmalloc::Heap::allocateMediumLineSlowCase): (bmalloc::Heap::allocateXLarge): (bmalloc::Heap::deallocateXLarge): (bmalloc::Heap::allocateLarge): (bmalloc::Heap::deallocateLarge): * bmalloc/Heap.h: Added. (bmalloc::Heap::deallocateSmallLine): (bmalloc::Heap::allocateSmallLine): (bmalloc::Heap::deallocateMediumLine): (bmalloc::Heap::allocateMediumLine): * bmalloc/Inline.h: Added. * bmalloc/LargeChunk.h: Added. (bmalloc::LargeChunk::begin): (bmalloc::LargeChunk::end): (bmalloc::LargeChunk::create): (bmalloc::LargeChunk::get): (bmalloc::LargeChunk::beginTag): (bmalloc::LargeChunk::endTag): * bmalloc/Line.h: Added. (bmalloc::Line::begin): (bmalloc::Line::end): (bmalloc::Line::concurrentRef): (bmalloc::Line::deref): * bmalloc/MediumAllocator.h: Added. (bmalloc::MediumAllocator::isNull): (bmalloc::MediumAllocator::MediumAllocator): (bmalloc::MediumAllocator::line): (bmalloc::MediumAllocator::allocate): (bmalloc::MediumAllocator::derefCount): (bmalloc::MediumAllocator::refill): * bmalloc/MediumChunk.h: Added. * bmalloc/MediumLine.h: Added. * bmalloc/MediumPage.h: Added. * bmalloc/MediumTraits.h: Added. * bmalloc/Mutex.cpp: Added. (bmalloc::Mutex::lockSlowCase): * bmalloc/Mutex.h: Added. (bmalloc::Mutex::Mutex): (bmalloc::Mutex::try_lock): (bmalloc::Mutex::lock): (bmalloc::Mutex::unlock): * bmalloc/ObjectType.cpp: Added. (bmalloc::objectType): * bmalloc/ObjectType.h: Added. (bmalloc::isSmallOrMedium): (bmalloc::isSmall): * bmalloc/Page.h: Added. (bmalloc::Page::ref): (bmalloc::Page::deref): (bmalloc::Page::refCount): * bmalloc/PerProcess.h: Added. (bmalloc::PerProcess::mutex): (bmalloc::PerProcess::getFastCase): (bmalloc::PerProcess::get): (bmalloc::PerProcess::getSlowCase): * bmalloc/PerThread.h: Added. (bmalloc::PerThreadStorage::get): (bmalloc::PerThreadStorage::init): (bmalloc::PerThreadStorage::get): (bmalloc::PerThreadStorage::init): (bmalloc::PerThread::getFastCase): (bmalloc::PerThread::get): (bmalloc::PerThread::destructor): (bmalloc::PerThread::getSlowCase): * bmalloc/Range.h: Added. (bmalloc::Range::Range): (bmalloc::Range::begin): (bmalloc::Range::end): (bmalloc::Range::size): (bmalloc::Range::operator!): (bmalloc::Range::operator<): * bmalloc/SegregatedFreeList.cpp: Added. (bmalloc::SegregatedFreeList::SegregatedFreeList): (bmalloc::SegregatedFreeList::insert): (bmalloc::SegregatedFreeList::takeGreedy): (bmalloc::SegregatedFreeList::take): * bmalloc/SegregatedFreeList.h: Added. * bmalloc/Sizes.h: Added. * bmalloc/SmallAllocator.h: Added. (bmalloc::SmallAllocator::isNull): (bmalloc::SmallAllocator::canAllocate): (bmalloc::SmallAllocator::SmallAllocator): (bmalloc::SmallAllocator::line): (bmalloc::SmallAllocator::allocate): (bmalloc::SmallAllocator::objectCount): (bmalloc::SmallAllocator::derefCount): (bmalloc::SmallAllocator::refill): * bmalloc/SmallChunk.h: Added. * bmalloc/SmallLine.h: Added. * bmalloc/SmallPage.h: Added. * bmalloc/SmallTraits.h: Added. * bmalloc/Syscall.h: Added. * bmalloc/VMAllocate.h: Added. (bmalloc::vmSize): (bmalloc::vmValidate): (bmalloc::vmAllocate): (bmalloc::vmDeallocate): (bmalloc::vmDeallocatePhysicalPages): (bmalloc::vmAllocatePhysicalPages): (bmalloc::vmDeallocatePhysicalPagesSloppy): (bmalloc::vmAllocatePhysicalPagesSloppy): * bmalloc/VMHeap.cpp: Added. (bmalloc::VMHeap::VMHeap): (bmalloc::VMHeap::allocateSmallChunk): (bmalloc::VMHeap::allocateMediumChunk): (bmalloc::VMHeap::allocateLargeChunk): * bmalloc/VMHeap.h: Added. (bmalloc::VMHeap::allocateSmallPage): (bmalloc::VMHeap::allocateMediumPage): (bmalloc::VMHeap::allocateLargeRange): (bmalloc::VMHeap::deallocateSmallPage): (bmalloc::VMHeap::deallocateMediumPage): (bmalloc::VMHeap::deallocateLargeRange): * bmalloc/Vector.h: Added. (bmalloc::Vector::begin): (bmalloc::Vector::end): (bmalloc::Vector::size): (bmalloc::Vector::capacity): (bmalloc::Vector::last): (bmalloc::Vector::pop): (bmalloc::Vector::Vector): (bmalloc::Vector::~Vector): (bmalloc::Vector::operator): (bmalloc::Vector::push): (bmalloc::Vector::pop): (bmalloc::Vector::shrink): (bmalloc::Vector::reallocateBuffer): (bmalloc::Vector::shrinkCapacity): (bmalloc::Vector::growCapacity): * bmalloc/XLargeChunk.h: Added. (bmalloc::XLargeChunk::get): (bmalloc::XLargeChunk::begin): (bmalloc::XLargeChunk::XLargeChunk): (bmalloc::XLargeChunk::create): (bmalloc::XLargeChunk::destroy): (bmalloc::XLargeChunk::range): (bmalloc::XLargeChunk::size): * bmalloc/bmalloc.h: Added. (bmalloc::api::malloc): (bmalloc::api::free): (bmalloc::api::realloc): * bmalloc/mbmalloc.cpp: Added.