====== Non-blocking Writes ====== ===== Participants ===== * [[http://www.cs.fiu.edu/~lusec001/|Luis Useche]] * [[|Jesus Ramos]] * [[http://www.cs.fiu.edu/~rkoll001/|Ricardo Koller]] * [[http://www.cs.fiu.edu/~dcamp020/|Daniel Campello]] * [[http://www.cs.fiu.edu/~raju/|Raju Rangaswami]] * [[http://domino.research.ibm.com/comm/research_people.nsf/pages/akshat_verma.index.html|Akshat Verma]] ===== Project Goals ===== (Truly) Non-blocking Writes! ===== ToDo List ===== === TODO's from 12/13/12 meeting === - Setup the key-value store benchmark (YCSB) to test NBW (file system path). - Fix the CPU scheduler bug that could potentially influence the performance degradation shown by some of the benchmarks. - Setup the consolidated benchmark using qemu on a Linux host. Two possible options here: * Use NBW at the host level and virtualize guest machines without KVM to create memory pressure at the hyper-visor level. * Use vanilla host and virtualize guest machines using KVM and let NBW work at the guest level. * We plan to set the base comparison level by partitioning statically the physical memory among all the VMs to let fit in memory each working set without problems. - Incrementally we increase the amount of VMs (i.e workloads) so they start competing for memory. We measure the improvements in execution times of NBW compared to the vanilla system. - The result of the benchmark is a vector of execution times where each dimension is the execution time of each workload in the consolidated system. There should be one vector for vanilla and NBW respectively. - Depending on time and the results of the key-value benchmark as a file system benchmark, explore other file systems benchmarks as databases or Filebench (possible using MongoDB as back-end) - With very low priority, we could rerun the SPEC SFS benchmark to see if there is any improvement by the inclusion of the read-from-patches implementation. === Future ToDo's === * Get times of changing from user to kernel mode, and the other way around. * Test the system changing queue depths with NCQ. * Explore creation of persistent patches. * Implement NFS support. ===== Meetings ===== * 01/08/15: Non-blocking write curves for paper {{{{internal:projects:aw:aw010815.mp3|mp3}} * 12/16/14: Handling of journaling modes; Responding to shepherd; Simulation of cache misses for partial page writes (Hector) {{{{internal:projects:aw:aw121614.mp3|mp3}} * 12/09/14: FAST'15 Review Comments (Audio lost) {{{{internal:projects:aw:aw120914.mp3|mp3}} * 11/25/14: Intro feedback and trace-replayed brainstorming {{{{internal:projects:aw:aw112514.mp3|mp3}} * 11/18/14: Writing Feedback/Task; Trace-replay updates {{{{internal:projects:aw:aw111814.mp3|mp3}} * 11/12/14: Writing / development schedule {{{{internal:projects:aw:aw111214.mp3|mp3}} * 11/04/14: Brainstorming: Concrete paper plan "fast durable file writes to persistent memory"{{{{internal:projects:aw:aw110414.mp3|mp3}} * 10/28/14: Brainstorming: fast durable file writes to persistent memory {{{{internal:projects:aw:aw102814.mp3|mp3}} * 10/21/14: Tracing and Persistent Memory direction {{{{internal:projects:aw:aw102114.mp3|mp3}} * 10/14/14: Next evaluation steps; tracing work; Project ideas (persistent memory) {{{{internal:projects:aw:aw101414.mp3|mp3}} * 10/07/14: Brainstorming next paper and projects {{{{internal:projects:aw:aw100714.mp3|mp3}} * 09/16/14: Module data; sqlite updates; other eval plans {{{{internal:projects:aw:aw091614.mp3|mp3}} * 09/12/14: New data / module / Eval/Writing plans {{{{internal:projects:aw:aw091214.mp3|mp3}} * 09/09/14: Eval/Writing plans {{{{internal:projects:aw:aw090914.mp3|mp3}} * 09/02/14: Kernel module discussion, Paper writing plans and new data with mobigen trace replay{{{{internal:projects:aw:aw090214.mp3|mp3}} * 08/26/14: Mobigen data (Facebook/Twitter) and plans for Mobigen replay {{{{internal:projects:aw:aw082614.mp3|mp3}} * 08/19/14: More updates on module, SQLite, Mobibench, Filebench {{{{internal:projects:aw:aw081914.mp3|mp3}} * 08/12/14: Updates on module, SQLite, Mobibench, Filebench {{{{internal:projects:aw:aw081214.mp3|mp3}} * 08/05/14: 4 Directions for motivation (module-based data), persistent-memory, trace analysis, new benchmarks {{{{internal:projects:aw:aw080514.mp3|mp3}} * 07/22/14: Byte addressable memory cache (new motivation); kernel module plans {{{{internal:projects:aw:aw072214.mp3|mp3}} * 07/15/14: Kernel module plans and new data collection {{{{internal:projects:aw:aw071514.mp3|mp3}} * 07/08/14: OSDI'14 rebuttal {{{{internal:projects:aw:aw070814.mp3|mp3}} * 07/01/14: Analyzing blktrace data; new I/O scheduling improvements; NCQ setup by Hector {{{{internal:projects:aw:aw070114.mp3|mp3}} * 06/24/14: Next steps: CPU nicing for kernel-threads issuing NBW + updating filebench graphs {{{{internal:projects:aw:aw062414.mp3|mp3}} * 06/10/14: Next steps: Limiting I/O queueing for non-blocking fetches and moving I/Os across priority queues when status changes to blocking {{{{internal:projects:aw:aw061014.mp3|mp3}} * 06/06/14: I/O Scheduler implementation update; next steps {{{{internal:projects:aw:aw060614.mp3|mp3}} * 05/27/14: I/O Scheduler Optimization --- Initial thoughts on implementation {{{{internal:projects:aw:aw052714.mp3|mp3}} * 04/24/14: Paper story-line I {{{{internal:projects:aw:aw042414.mp3|mp3}} * 04/22/14: Evaluation story-line and discussion of experiments {{{{internal:projects:aw:aw042214.mp3|mp3}} * 04/17/14: Evaluation outline/goals {{{{internal:projects:aw:aw041714.mp3|mp3}} * 04/15/14: New SpecSFS data is promising {{{{internal:projects:aw:aw041514.mp3|mp3}} * 03/20/14: OSDI paper discussion {{{{internal:projects:aw:aw032014.mp3|mp3}} * 03/04/14: New data with lazy; HotStorage paper? {{{{internal:projects:aw:aw030414.mp3|mp3}} * 02/13/14: Potential workloads for file system {{{{internal:projects:aw:aw021314.mp3|mp3}} * 02/06/14: Sawtooth performance explained and next steps for experiments {{{{internal:projects:aw:aw020614.mp3|mp3}} * 01/28/14: SMP works! Kernel Preemption works! Explaining sawtooth performance; {{{{internal:projects:aw:aw012814.mp3|mp3}} * 01/23/14: Understanding performance data {{{{internal:projects:aw:aw012314.mp3|mp3}} * 01/21/14: Understanding performance data {{{{internal:projects:aw:aw012114.mp3|mp3}} * 01/14/14: First great set of Filebench data!, YCSB+LevelDB? ATC submission plans {{{{internal:projects:aw:aw011414.mp3|mp3}} * 12/11/13: Implementation updates, review of paper outline {{{{internal:projects:aw:aw121113.mp3|mp3}} * 12/04/13: Paper outline (First draft) and discuss of current bugs {{{{internal:projects:aw:aw120413.mp3|mp3}} * 11/27/13: Bugs fixed, discussion about implementing pdflush, fsync, and lazy, paper outline and writing tasks {{{{internal:projects:aw:aw112713.mp3|mp3}} * 11/20/13: Two bugs, 1 solution and storyline update {{{{internal:projects:aw:aw112013.mp3|mp3}} * 11/13/13: Brief update {{{{internal:projects:aw:aw111313.mp3|mp3}} * 11/06/13: Story-line draft / scheduled implementation / deadlock solved {{{{internal:projects:aw:aw110613.mp3|mp3}} * 10/30/13: Deadlock! {{{{internal:projects:aw:aw103013.mp3|mp3}} * 10/23/13: Updates{{{{internal:projects:aw:aw102313.mp3|mp3}} * 10/16/13: FS-specific metadata is cached via page cache using device mapping! {{{{internal:projects:aw:aw101613.mp3|mp3}} * 10/11/13: Two options to explore for FS metadata blocking (1) lazy page fetching and (2) work-queue based threaded page fetching {{{{internal:projects:aw:aw101113.mp3|mp3}} * 10/09/13: Sync metadata access seems to block NBW; discussion of next steps {{{{internal:projects:aw:aw100913.mp3|mp3}} * 09/25/13: Seems like non-blocking writes are blocking! {{{{internal:projects:aw:aw092513.mp3|mp3}} * 09/23/13: Results examination for lazy, planning for filebench runs {{{{internal:projects:aw:aw092313.mp3|mp3}} * 09/20/13: Results examination, development, and paper writing tasks {{{{internal:projects:aw:aw092013.mp3|mp3}} * 09/18/13: Inspection of NFS client kernel-code {{{{internal:projects:aw:aw091813.mp3|mp3}} * 09/16/13: Results for SPECSfs and Filebench, updates on NFS client {{{{internal:projects:aw:aw091613.mp3|mp3}} * 09/13/13: Updates on filebench (fileserver) and SPECSfs {{{{internal:projects:aw:aw091313.mp3|mp3}} * 09/11/13: New workload plan -- jdbc,jdbm,gdbm(java), etc. {{{{internal:projects:aw:aw091113.mp3|mp3}} * 09/06/13: New evaluation plan {{{{internal:projects:aw:aw090613.mp3|mp3}} * 08/28/13: Next steps: SpecSFS anaysis, I/O scheduler, other FS benchmarks {{{{internal:projects:aw:aw082813.mp3|mp3}} * 08/21/13: Updates on SPECSfs behavior and controlling runtime {{{{internal:projects:aw:aw082113.mp3|mp3}} * 08/19/13: Updates/discussion on NFS client -- does it block on write? {{{{internal:projects:aw:aw081913.mp3|mp3}} * 08/14/13: Implementation update for CPU scheduler and some initial direction of NFS implementation; Results update (Aldo) {{{{internal:projects:aw:aw081413.mp3|mp3}} * 08/09/13: NFS server - async config, many updates from Daniel on development - NetApp project {{{{internal:projects:aw:aw080913.mp3|mp3}} * 07/31/13: Memory ordering discussion + updates from Aldo {{{{internal:projects:aw:aw073113.mp3|mp3}} * 07/24/13: I/O prioritization v.s. Lazy file system implementation + discussion of next set of configurations to run {{{{internal:projects:aw:aw072413.mp3|mp3}} * 07/17/13: Review of current status (Aldo, Daniel, Luis) and thoughts on syscall tracing {{{{internal:projects:aw:aw071713.mp3|mp3}} * 07/10/13: Discussion with Luis about FAST paper and Filesystem oriented submission {{{{internal:projects:aw:aw071013.mp3|mp3}} * 07/03/13: Discussion with Luis on implementation/documentation next steps {{{{internal:projects:aw:aw070313.mp3|mp3}} * 04/29/13: FIO new data points to NCQ being effective -- need to test with larger file size in FIO {{{{internal:projects:aw:aw042913.mp3|mp3}} * 04/23/13: NCQ results indicate fake-ncq -- Aldo will confirm with more expts; New results with Lazy (10K max patches) are promising {{{{internal:projects:aw:aw042313.mp3|mp3}} * 04/18/13: Three directions next: reliability/reproducability of data, NCQ, and Jesus' scheduler completion and integration with Lazy {{{{internal:projects:aw:aw041813.mp3|mp3}} * 04/11/13: Two primary strengths of NBW not capitalized in current implementation {{{{internal:projects:aw:aw041113.mp3|mp3}} * 03/28/13: Strategy discussion {{{{internal:projects:aw:aw032813.mp3|mp3}} * 03/26/13: New data -- higher queue depths not proportional to fraction of NBW requests (request merging the answer?) (as single-stepping) {{{{internal:projects:aw:aw032613.mp3|mp3}} * 03/21/13: Potential source of failure identified (as single-stepping) {{{{internal:projects:aw:aw032113.mp3|mp3}} * 03/19/13: Failure issues with dacapo {{{{internal:projects:aw:aw031913.mp3|mp3}} * 01/31/13: Initial set of results with new CPU scheduler; next steps fix cpu scheduler optimization for mmap faulting processes; correlating counters with performance; I/O scheduler optimization; memory corruption issue {{{{internal:projects:aw:aw013113.mp3|mp3}} * 01/17/13: Updates and status of CPU scheduler implementation from Jesus {{{{internal:projects:aw:aw011713.mp3|mp3}} * 01/15/13: Memory sensitivity and benchmarks for evaluation {{{{internal:projects:aw:aw011513.mp3|mp3}} * 12/13/12: Evaluation plan revision [[internal:projects:aw:summary21312:start|(Meeting summary)]] (No audio) * 12/06/12: Updates from Daniel, reviewing remaining tasks for paper submission {{{{internal:projects:aw:aw120612.mp3|mp3}} * 11/28/12: Fixed state-assignment bug in CPU scheduler; Dealing with NCQ queue lengths; experiments review {{{{internal:projects:aw:aw112812.mp3|mp3}} * 11/21/12: Hunting NCQ hardware and getting NCQ data for NBW, plans for knowledge transfer to Daniel {{{{internal:projects:aw:aw112112.mp3|mp3}} * 11/14/12: Updates on validating I/O priority inversion issue and VFS implementation, benchmarking and evaluation plans for paper submission {{{{internal:projects:aw:aw111412.mp3|mp3}} * 10/31/12: I/O scheduler optimizations and handling device-level queues; svnserver benchmark has negligible WriteFaults {{{{internal:projects:aw:aw103112.mp3|mp3}} * 10/24/12: Updates on VFS implementation {{{{internal:projects:aw:aw102412.mp3|mp3}} * 10/17/12: Updates on benchmarks and Do we need an I/O priority scheduler for NBW processes? {{{{internal:projects:aw:aw101712.mp3|mp3}} * 10/09/12: More updates on scheduler and benchmarks {{{{internal:projects:aw:aw100912.mp3|mp3}} * 10/03/12: Scheduling issue performance degradation, benchmark updates {{{{internal:projects:aw:aw100312.mp3|mp3}} * 09/26/12: Scheduling fixes and new results, anomaly with batik and pmd which have severe degradation {{{{internal:projects:aw:aw092612.mp3|mp3}} * 09/13/12: Plan and action items when Luis is absent {{{{internal:projects:aw:aw091312.mp3|mp3}} * 09/05/12: New data motivates scheduling as the next step, updates on NBW overhead breakdown {{{{internal:projects:aw:aw090512.mp3|mp3}} * 08/23/12: Understanding new data and next steps with implementation and benchmarks {{{{internal:projects:aw:aw082312.mp3|mp3}} * 08/16/12: Update on new CPU scheduler (almost done!) and reviewing evaluation plan {{{{internal:projects:aw:aw081612.mp3|mp3}} * 08/07/12: Prioritizing next set of action items {{{{internal:projects:aw:aw080712.mp3|mp3}} * 08/02/12: New evaluation plan {{{{internal:projects:aw:aw080212.mp3|mp3}} * 07/26/12: Discussing remaining OSDI feedback {{{{internal:projects:aw:aw072612.mp3|mp3}} * 07/24/12: New results with NBW scheduling optimizations + Discussing OSDI feedback {{{{internal:projects:aw:aw072412.mp3|mp3}} * 07/17/12: NBW CPU scheduler with new state + plans for collecting results from Jesus {{{{internal:projects:aw:aw071712.mp3|mp3}} * 07/10/12: Results review and new CPU yielding implementation (for Jesus) {{{{internal:projects:aw:aw071012.mp3|mp3}} * 07/03/12: NSF proposal comments & CPU scheduling optimization {{{{internal:projects:aw:aw070312.mp3|mp3}} * 06/26/12: Results with multi-threaded apps lackluster; yield optimization and SMP implementations; big picture of I/O and CPU parallelism in NBW {{{{internal:projects:aw:aw062612.mp3|mp3}} * 06/21/12: Bug fixes, new results, Luis thesis chapter feedback {{{{internal:projects:aw:aw062112.mp3|mp3}} * 06/12/12: Scheduled fetch design and async fetch results {{{{internal:projects:aw:aw061212.mp3|mp3}} * 04/10/12: New directions {{{{internal:projects:aw:aw041012.mp3|mp3}} * 04/03/12: Perils of prefetching and plugging! {{{{internal:projects:aw:aw040312.mp3|mp3}} * 04/02/12: Paper structure, stories, and data analysis {{{{internal:projects:aw:aw040212.mp3|mp3}} * 03/27/12: Disassembly doesn't seem to help with big-patch-queue! -- Alternate evaluation plan {{{{internal:projects:aw:aw032712.mp3|mp3}} * 03/20/12: Update from Jesus on new disassembly implementation (to replace single-stepping) {{{{internal:projects:aw:aw032012.mp3|mp3}} * 03/06/12: More data and more analysis {{{{internal:projects:aw:aw030612.mp3|mp3}} * 03/02/12: Alternate explanations for data {{{{internal:projects:aw:aw030212.mp3|mp3}} * 02/28/12: Analyzing initial detailed data from benchmarks {{{{internal:projects:aw:aw022812.mp3|mp3}} * 02/23/12: Understanding minor fault and patch creation latencies {{{{internal:projects:aw:aw022312.mp3|mp3}} * 02/21/12: Explaining the data III {{{{internal:projects:aw:aw022112.mp3|mp3}} * 02/07/12: Request and page pools and ideas for deferred memory allocation {{{{internal:projects:aw:aw020712.mp3|mp3}} * 01/31/12: Trying out reboot/power-down and tracing blktrace problems {{{{internal:projects:aw:aw013112.mp3|mp3}} * 01/24/12: Explaining the data II {{{{internal:projects:aw:aw012412.mp3|mp3}} * 01/17/12: Explaining the data I {{{{internal:projects:aw:aw011712.mp3|mp3}} * 01/10/12: Discussion of issues with implementation and initial data {{{{internal:projects:aw:aw011012.mp3|mp3}} * 12/13/11: Updates and plans for new experiments (PCIe flash, memory sizing, SMP) {{{{internal:projects:aw:aw121311.mp3|mp3}} * 12/06/11: Experiment updates; some FS performance analysis; {{{{internal:projects:aw:aw120611.mp3|mp3}} * 11/29/11: Issues uncovered during merging of anon and fs {{{{internal:projects:aw:aw112911.mp3|mp3}} * 11/22/11: NSF Proposal Discussion {{{{internal:projects:aw:aw112211.mp3|mp3}} * 11/15/11: Brainstorming ideas for debugging filesystem component {{{{internal:projects:aw:aw111511.mp3|mp3}} * 11/08/11: Page fault handling - Normal and NBW versions - details for the paper & Experiment updates {{{{internal:projects:aw:aw110811.mp3|mp3}} * 11/01/11: Status updates {{{{internal:projects:aw:aw110111.mp3|mp3}} * 10/18/11: Discussion of statistics for analyzing performance {{{{internal:projects:aw:aw101811.mp3|mp3}} * 10/11/11: More performance testing of anonymous version {{{{internal:projects:aw:aw101111.mp3|mp3}} * 10/04/11: Initial performance of anonymous -- promising results {{{{internal:projects:aw:aw100411.mp3|mp3}} * 09/20/11: Can we make FAST? {{{{internal:projects:aw:aw092011.mp3|mp3}} * 09/13/11: Implementation progress {{{{internal:projects:aw:aw091311.mp3|mp3}} * 09/06/11: update from Jesus, FS implementation, initial writing plans {{{{internal:projects:aw:aw090611.mp3|mp3}} * 08/30/11: Implementing patch application in process context, workloads for non-blocking FS writes, Deferred/No async read, {{{{internal:projects:aw:aw083011.mp3|mp3}} * 08/16/11: Updates {{{{internal:projects:aw:aw081611.mp3|mp3}} * 08/02/11: War of the Titans II - Disassembly vs. page diff-merge (and some very good advice on research prototyping too!) {{{{internal:projects:aw:aw080211.mp3|mp3}} * 07/26/11: using page flags and page-level locking {{{{internal:projects:aw:aw072611.mp3|mp3}} * 07/19/11: setting up async reads and other updates {{{{internal:projects:aw:aw071911.mp3|mp3}} * 07/12/11: handling async read completions, handling write syscalls {{{{internal:projects:aw:aw071211.mp3|mp3}} * 06/07/11: 32 vs. 64 bit porting of memcheck.c -- next steps: porting to 64 bit, random single-step checks, radix/range/redblack tree based patch implementation {{{{internal:projects:aw:aw060711.mp3|mp3}} * 05/31/11: More generic w/ generic_fault_handler; thoughts on consistency and syscall implementation {{{{internal:projects:aw:aw053111.mp3|mp3}} * 05/23/11: Updates on patch implementation {{{{internal:projects:aw:aw052311.mp3|mp3}} * 05/17/11: Single stepping v.s. disassembly for write size {{{{internal:projects:aw:aw051711.mp3|mp3}} * 05/10/11: Initial thoughts on handling syscalls {{{{internal:projects:aw:aw051011.mp3|mp3}} * 05/03/11: Initial kernel design {{{{internal:projects:aw:aw050311.mp3|mp3}} * 04/19/11: Addressing Hotstorage review comments {{{{internal:projects:aw:aw041911.mp3|mp3}} * 03/11/11: Result review and Paper writing plans. {{{{internal:projects:aw:aw031111.mp3|mp3}} * 03/09/11: (evening skype call) Review {{{{internal:projects:aw:aw030911b.mp3|mp3}} * 03/09/11: Deconstructing results, motivation discussion, and changes to graph data {{{{internal:projects:aw:aw030911.mp3|mp3}} * 03/04/11: Hotstorage paper structure {{{{internal:projects:aw:aw030411.mp3|mp3}}