DiskLruCache 能够正常工作是依赖于 journal 文件中的内容,journal 文件中会存储每次读取操作的记录。
我们来看看 journal 文件中的内容是什么样的吧,如下所示
libcore.io.DiskLruCache
1
1
1
DIRTY 27c7e00adbacc71dc793e5e7bf02f861
CLEAN 27c7e00adbacc71dc793e5e7bf02f861 1208
READ 27c7e00adbacc71dc793e5e7bf02f861
DIRTY b80f9eec4b616dc6682c7fa8bas2061f
CLEAN b80f9eec4b616dc6682c7fa8bas2061f 1208
READ b80f9eec4b616dc6682c7fa8bas2061f
DIRTY be3fgac81c12a08e89088555d85dfd2b
CLEAN be3fgac81c12a08e89088555d85dfd2b 99
READ be3fgac81c12a08e89088555d85dfd2b
DIRTY 536990f4dbddfghcfbb8f350a941wsxd
REMOVE 536990f4dbddfghcfbb8f350a941wsxd
来看一下前五行:
前五行是该文件的文件头,DiskLruCache 初始化的时候,如果该文件存在,就需要校验该文件头。
下面来看各个状态的含义:
第六行后的分析
源码如下所示
public static DiskLruCache open(File directory, int appVersion, int valueCount, long maxSize)
throws IOException {
if (maxSize <= 0) {
throw new IllegalArgumentException("maxSize <= 0");
}
if (valueCount <= 0) {
throw new IllegalArgumentException("valueCount <= 0");
}
// If a bkp file exists, use it instead.
File backupFile = new File(directory, JOURNAL_FILE_BACKUP);
if (backupFile.exists()) {
File journalFile = new File(directory, JOURNAL_FILE);
// If journal file also exists just delete backup file.
if (journalFile.exists()) {
backupFile.delete();
} else {
renameTo(backupFile, journalFile, false);
}
}
// Prefer to pick up where we left off.
DiskLruCache cache = new DiskLruCache(directory, appVersion, valueCount, maxSize);
if (cache.journalFile.exists()) {
try {
cache.readJournal();
cache.processJournal();
return cache;
} catch (IOException journalIsCorrupt) {
System.out
.println("DiskLruCache "
+ directory
+ " is corrupt: "
+ journalIsCorrupt.getMessage()
+ ", removing");
cache.delete();
}
}
// Create a new empty cache.
directory.mkdirs();
cache = new DiskLruCache(directory, appVersion, valueCount, maxSize);
cache.rebuildJournal();
return cache;
}
首先检查 journal 的备份文件是否存在( journal.bkp),如果备份存在,然后检查 journal 文件是否存在,如果 journal 文件存在,备份文件就可以删除了;如果 journal 文件不存在,将备份文件文件重命名为 journal 文件。
然后检查 journal 文件是否存在,如果不存在,创建 directory,重新构造 DiskLruCache,再调用 rebuildJournal() 建立 journal 文件。
/**
* Creates a new journal that omits redundant information. This replaces the
* current journal if it exists.
*/
private synchronized void rebuildJournal() throws IOException {
if (journalWriter != null) {
journalWriter.close();
}
Writer writer = new BufferedWriter(
new OutputStreamWriter(new FileOutputStream(journalFileTmp), Util.US_ASCII));
try {
writer.write(MAGIC);
writer.write("\n");
writer.write(VERSION_1);
writer.write("\n");
writer.write(Integer.toString(appVersion));
writer.write("\n");
writer.write(Integer.toString(valueCount));
writer.write("\n");
writer.write("\n");
for (Entry entry : lruEntries.values()) {
if (entry.currentEditor != null) {
writer.write(DIRTY + ' ' + entry.key + '\n');
} else {
writer.write(CLEAN + ' ' + entry.key + entry.getLengths() + '\n');
}
}
} finally {
writer.close();
}
if (journalFile.exists()) {
renameTo(journalFile, journalFileBackup, true);
}
renameTo(journalFileTmp, journalFile, false);
journalFileBackup.delete();
journalWriter = new BufferedWriter(
new OutputStreamWriter(new FileOutputStream(journalFile, true), Util.US_ASCII));
}
可以看到首先构建一个 journal.tmp 文件,然后写入文件头(5行),然后遍历 lruEntries( LinkedHashMap lruEntries = new LinkedHashMap(0, 0.75f, true); ),此时 lruEntries 里没有任何数据。接下来将 tmp 文件重命名为 journal 文件。这样一个 journal 文件便生成了。如果存在,那么调用 readJournal()
private void readJournal() throws IOException {
StrictLineReader reader = new StrictLineReader(new FileInputStream(journalFile), Util.US_ASCII);
try {
String magic = reader.readLine();
String version = reader.readLine();
String appVersionString = reader.readLine();
String valueCountString = reader.readLine();
String blank = reader.readLine();
if (!MAGIC.equals(magic)
|| !VERSION_1.equals(version)
|| !Integer.toString(appVersion).equals(appVersionString)
|| !Integer.toString(valueCount).equals(valueCountString)
|| !"".equals(blank)) {
throw new IOException("unexpected journal header: [" + magic + ", " + version + ", "
+ valueCountString + ", " + blank + "]");
}
int lineCount = 0;
while (true) {
try {
readJournalLine(reader.readLine());
lineCount++;
} catch (EOFException endOfJournal) {
break;
}
}
redundantOpCount = lineCount - lruEntries.size();
// If we ended on a truncated line, rebuild the journal before appending to it.
if (reader.hasUnterminatedLine()) {
rebuildJournal();
} else {
journalWriter = new BufferedWriter(new OutputStreamWriter(
new FileOutputStream(journalFile, true), Util.US_ASCII));
}
} finally {
Util.closeQuietly(reader);
}
}
首先校验文件头,接下来调用 readJournalLine 按行读取内容。
private void readJournalLine(String line) throws IOException {
int firstSpace = line.indexOf(' ');
if (firstSpace == -1) {
throw new IOException("unexpected journal line: " + line);
}
int keyBegin = firstSpace + 1;
int secondSpace = line.indexOf(' ', keyBegin);
final String key;
if (secondSpace == -1) {
key = line.substring(keyBegin);
if (firstSpace == REMOVE.length() && line.startsWith(REMOVE)) {
lruEntries.remove(key);
return;
}
} else {
key = line.substring(keyBegin, secondSpace);
}
Entry entry = lruEntries.get(key);
if (entry == null) {
entry = new Entry(key);
lruEntries.put(key, entry);
}
if (secondSpace != -1 && firstSpace == CLEAN.length() && line.startsWith(CLEAN)) {
String[] parts = line.substring(secondSpace + 1).split(" ");
entry.readable = true;
entry.currentEditor = null;
entry.setLengths(parts);
} else if (secondSpace == -1 && firstSpace == DIRTY.length() && line.startsWith(DIRTY)) {
entry.currentEditor = new Editor(entry);
} else if (secondSpace == -1 && firstSpace == READ.length() && line.startsWith(READ)) {
// This work was already done by calling lruEntries.get().
} else {
throw new IOException("unexpected journal line: " + line);
}
}
journal 日志每一行中的各个部分都是用 ' ' 空格来分割的,所以先用 空格来截取一下。
一般正常操作下 DIRTY 不会单独出现,会和 REMOVE or CLEAN 成对出现;经过上面这个流程,基本上加入到 lruEntries 里面的只有 CLEAN 且没有被 REMOVE 的 key。
然后我们回到 readJournal 方法,在我们按行读取的时候,会记录一下 lineCount ,然后赋值给 redundantOpCount ,该变量记录的是多余的记录条数( redundantOpCount = lineCount - lruEntries.size(); 文件的行数-真正可以的 key 的行数)。最后,如果读取过程中发现 journal 文件有问题,则重建 journal 文件。没有问题的话,初始化下 journalWriter,关闭 reader。
我们再回到 open() 方法中,readJournal 完成了,会继续调用 processJournal() 这个方法
/**
* Computes the initial size and collects garbage as a part of opening the
* cache. Dirty entries are assumed to be inconsistent and will be deleted.
*/
private void processJournal() throws IOException {
deleteIfExists(journalFileTmp);
for (Iterator<Entry> i = lruEntries.values().iterator(); i.hasNext(); ) {
Entry entry = i.next();
if (entry.currentEditor == null) {
for (int t = 0; t < valueCount; t++) {
size += entry.lengths[t];
}
} else {
entry.currentEditor = null;
for (int t = 0; t < valueCount; t++) {
deleteIfExists(entry.getCleanFile(t));
deleteIfExists(entry.getDirtyFile(t));
}
i.remove();
}
}
}
计算缓存初始大小,赋值给 size,并收集垃圾作为打开缓存的一部分。对于所有非法 DIRTY 状态(就是 DIRTY 单独出现的)的 entry,如果存在 文件则删除,并且从 lruEntries 中移除。此时,剩下的就只有 CLEAN 状态的 key 记录了。
到此 DiskLruCache 就初始化完毕了,为了方便记忆,我们来捋一下流程:
源码如下所示
public Editor edit(String key) throws IOException {
return edit(key, ANY_SEQUENCE_NUMBER);
}
private synchronized Editor edit(String key, long expectedSequenceNumber) throws IOException {
checkNotClosed();
validateKey(key);
Entry entry = lruEntries.get(key);
if (expectedSequenceNumber != ANY_SEQUENCE_NUMBER && (entry == null
|| entry.sequenceNumber != expectedSequenceNumber)) {
return null; // Snapshot is stale.
}
if (entry == null) {
entry = new Entry(key);
lruEntries.put(key, entry);
} else if (entry.currentEditor != null) {
return null; // Another edit is in progress.
}
Editor editor = new Editor(entry);
entry.currentEditor = editor;
// Flush the journal before creating files to prevent file leaks.
journalWriter.write(DIRTY + ' ' + key + '\n');
journalWriter.flush();
return editor;
}
private void validateKey(String key) {
Matcher matcher = LEGAL_KEY_PATTERN.matcher(key);
if (!matcher.matches()) {
throw new IllegalArgumentException("keys must match regex "
+ STRING_KEY_PATTERN + ": \"" + key + "\"");
}
}
private void checkNotClosed() {
if (journalWriter == null) {
throw new IllegalStateException("cache is closed");
}
}
首先验证 key,必须是由字母、数字、下划线、横线(-) 组成,且长度在 1-120 之间。
然后通过 key 获取实体 Entry ,如果 Entry 不存在,则创建一个 Entry 加入到 lruEntries 中;如果存在且不是正在编辑的实体,则直接使用。然后为 entry.currentEditor 进行赋值为 new Editor(entry); ,最后在 journal 文件中写入一条 DIRTY 记录,代表这个文件正在被操作。拿到 editor 对象以后,就是去调用 newOutputStream 去获得一个文件输入流了。
public OutputStream newOutputStream(int index) throws IOException {
if (index < 0 || index >= valueCount) {
throw new IllegalArgumentException("Expected index " + index + " to "
+ "be greater than 0 and less than the maximum value count "
+ "of " + valueCount);
}
synchronized (DiskLruCache.this) {
if (entry.currentEditor != this) {
throw new IllegalStateException();
}
if (!entry.readable) {
written[index] = true;
}
File dirtyFile = entry.getDirtyFile(index);
FileOutputStream outputStream;
try {
outputStream = new FileOutputStream(dirtyFile);
} catch (FileNotFoundException e) {
// Attempt to recreate the cache directory.
directory.mkdirs();
try {
outputStream = new FileOutputStream(dirtyFile);
} catch (FileNotFoundException e2) {
// We are unable to recover. Silently eat the writes.
return NULL_OUTPUT_STREAM;
}
}
return new FaultHidingOutputStream(outputStream);
}
}
首先校验 index 是否在 (0, valueCount] 范围内,一般我们使用都是一个 key 对应一个文件,所以传入的基本都是 0。接下来就是通过 entry.getDirtyFile(index); 拿到一个 dirtyFile 对象,其实就是个中转文件,文件格式为 key.index.tmp ,将这个文件的 FileOutputStream 通过 FaultHidingOutputStream 封装下传给我们。
最后,我们通过 outputStream 写入数据以后,需要调用 commit 方法
/**
* Commits this edit so it is visible to readers. This releases the
* edit lock so another edit may be started on the same key.
*/
public void commit() throws IOException {
if (hasErrors) {
completeEdit(this, false);
remove(entry.key); // The previous entry is stale.
} else {
completeEdit(this, true);
}
committed = true;
}
首先通过 hasErrors 判断,是否有错误发生,如果有就调用 completeEdit(this, false); 和 remove(entry.key); ;如果没有就调用 completeEdit(this, true); 。
下面看看 completeEdit 方法
private synchronized void completeEdit(Editor editor, boolean success) throws IOException {
Entry entry = editor.entry;
if (entry.currentEditor != editor) {
throw new IllegalStateException();
}
// If this edit is creating the entry for the first time, every index must have a value.
if (success && !entry.readable) {
for (int i = 0; i < valueCount; i++) {
if (!editor.written[i]) {
editor.abort();
throw new IllegalStateException("Newly created entry didn't create value for index " + i);
}
if (!entry.getDirtyFile(i).exists()) {
editor.abort();
return;
}
}
}
for (int i = 0; i < valueCount; i++) {
File dirty = entry.getDirtyFile(i);
if (success) {
if (dirty.exists()) {
File clean = entry.getCleanFile(i);
dirty.renameTo(clean);
long oldLength = entry.lengths[i];
long newLength = clean.length();
entry.lengths[i] = newLength;
size = size - oldLength + newLength;
}
} else {
deleteIfExists(dirty);
}
}
redundantOpCount++;
entry.currentEditor = null;
if (entry.readable | success) {
entry.readable = true;
journalWriter.write(CLEAN + ' ' + entry.key + entry.getLengths() + '\n');
if (success) {
entry.sequenceNumber = nextSequenceNumber++;
}
} else {
lruEntries.remove(entry.key);
journalWriter.write(REMOVE + ' ' + entry.key + '\n');
}
journalWriter.flush();
if (size > maxSize || journalRebuildRequired()) {
executorService.submit(cleanupCallable);
}
}
首先判断 if (success && !entry.readable) 是否成功,且是第一次写入(如果以前这个记录有值,则 readable = true ),内部的判断,我们都不会走,因为 written[i] 在 newOutputStream 的时候被写入 true 了。而且正常情况下,getDirtyFile 是存在的。
然后如果成功,将 dirtyFile 进行重命名为 cleanFile,文件名为:key.index。然后刷新 size 的长度。如果失败,则删除 dirtyFile .
然后,如果成功或者 readable 为 true ,将 readable 设置为 true ,写入一条 CLEAN 记录。如果第一次提交且失败,那么就会从 lruEntries.remove(key) ,写入一条 REMOVE 记录。
写入缓存,要判断是否超过了最大 size ,或者需要重建 journal 文件
/**
* We only rebuild the journal when it will halve the size of the journal
* and eliminate at least 2000 ops.
*/
private boolean journalRebuildRequired() {
final int redundantOpCompactThreshold = 2000;
return redundantOpCount >= redundantOpCompactThreshold //
&& redundantOpCount >= lruEntries.size();
}
如果 redundantOpCount 达到 2000,且超过了 lruEntries.size() 就会重建,这里就可以看到 redundantOpCount 的作用了。防止 journal 文件过大。到此我们的存入缓存就分析完成了。
总结:首先调用 editor,拿到指定的 dirtyFile 的 OutputStream ,然后开始写操作,写完后,记得调用 commit . commit 中会检测你是否发生 IOException ,若无,则将 dirtyFile -> cleanFile ,将 readable = true ,写入 CLEAN 记录。如若发生错误,则删除 dirtyFile ,从 lruEntries 中移除,然后写入一条 REMOVE 记录。
源码如下所示
public synchronized Snapshot get(String key) throws IOException {
checkNotClosed();
validateKey(key);
Entry entry = lruEntries.get(key);
if (entry == null) {
return null;
}
if (!entry.readable) {
return null;
}
// Open all streams eagerly to guarantee that we see a single published
// snapshot. If we opened streams lazily then the streams could come
// from different edits.
InputStream[] ins = new InputStream[valueCount];
try {
for (int i = 0; i < valueCount; i++) {
ins[i] = new FileInputStream(entry.getCleanFile(i));
}
} catch (FileNotFoundException e) {
// A file must have been deleted manually!
for (int i = 0; i < valueCount; i++) {
if (ins[i] != null) {
Util.closeQuietly(ins[i]);
} else {
break;
}
}
return null;
}
redundantOpCount++;
journalWriter.append(READ + ' ' + key + '\n');
if (journalRebuildRequired()) {
executorService.submit(cleanupCallable);
}
return new Snapshot(key, entry.sequenceNumber, ins, entry.lengths);
}
如果根据 key 取到的 Entry 为 null ,或者 readable = false ,则返回 null,否则将 cleanFile 的 FileInputStream 进行封装返回 Snapshot,且写入一条 READ 语句。然后 getInputStream 就是返回该 FileInputStream 了。
源码如下所示
/**
* Drops the entry for {@code key} if it exists and can be removed. Entries
* actively being edited cannot be removed.
*
* @return true if an entry was removed.
*/
public synchronized boolean remove(String key) throws IOException {
checkNotClosed();
validateKey(key);
Entry entry = lruEntries.get(key);
if (entry == null || entry.currentEditor != null) {
return false;
}
for (int i = 0; i < valueCount; i++) {
File file = entry.getCleanFile(i);
if (file.exists() && !file.delete()) {
throw new IOException("failed to delete " + file);
}
size -= entry.lengths[i];
entry.lengths[i] = 0;
}
redundantOpCount++;
journalWriter.append(REMOVE + ' ' + key + '\n');
lruEntries.remove(key);
if (journalRebuildRequired()) {
executorService.submit(cleanupCallable);
}
return true;
}
如果实体存在且不是正在被编辑,就可以直接进行删除,然后写入一条 REMOVE 记录。
如下所示代码
/**
* Closes this cache. Stored values will remain on the filesystem.
*/
public synchronized void close() throws IOException {
if (journalWriter == null) {
return; // already closed
}
for (Entry entry : new ArrayList<Entry>(lruEntries.values())) {
if (entry.currentEditor != null) {
entry.currentEditor.abort();
}
}
trimToSize();
journalWriter.close();
journalWriter = null;
}
这个方法用于将 DiskLruCache 关闭掉,关闭前,会判断所有正在编辑的实体,调用 abort() 方法,最后关闭 journalWriter 。close() 是和 open() 方法对应的一个方法。关闭之后就不能再调用 DiskLruCache 中任何操作缓存数据的方法,通常只应该在 Activity 的 onDestroy() 方法中去调用 close() 方法。
其中 abort() 方法是存储失败时的逻辑,中止此编辑。这将释放编辑锁,以便其他操作可以获取编辑锁,操作同一个 key。
/**
* Aborts this edit. This releases the edit lock so another edit may be
* started on the same key.
*/
public void abort() throws IOException {
completeEdit(this, false);
}
代码如下所示
/**
* Closes the cache and deletes all of its stored values. This will delete
* all files in the cache directory including files that weren't created by the cache.
*/
public void delete() throws IOException {
close();
IoUtils.deleteContents(directory);
}
这个方法用于将所有的缓存数据全部删除。比如 APP 中手动清理缓存功能,只需要调用一下 DiskLruCache 的 delete() 方法就可以实现了。
如下所示
/**
* Returns the number of bytes currently being used to store the values in
* this cache. This may be greater than the max size if a background
* deletion is pending.
*/
public synchronized long size() {
return size;
}
这个方法会返回当前缓存路径下所有缓存数据的总字节数,以 byte 为单位,如果应用程序中需要在界面上显示当前缓存数据的总大小,就可以通过调用这个方法计算出来。如果后台删除挂起,则此值可能大于最大大小。
代码如下所示
/**
* Force buffered operations to the filesystem.
*/
public synchronized void flush() throws IOException {
checkNotClosed();
trimToSize();
journalWriter.flush();
}
这个方法用于将内存中的操作记录同步到日志文件(也就是 journal 文件)当中。这个方法非常重要,因为 DiskLruCache 能够正常工作是依赖于 journal 文件中的内容。其实并不是每次写入缓存都要调用一次 flush() 方法,频繁地调用并不会带来任何好处,只会额外增加同步 journal 文件的时间。比较标准的做法是在 Activity 的 onPause() 方法中调用一次 flush() 方法就可以了。
至此,我们的源码分析就结束了。可以看到 DiskLruCache ,利用一个 journal 文件,保证了 cache 实体的可用性(只有CLEAN的可用),且获取文件的长度的时候可以通过在该文件的记录中读取。
利用 FaultHidingOutputStream 对 FileOutPutStream 在写入文件过程中是否发生错误进行捕获,而不是让用户手动去调用出错后的处理方法。其内部的很多细节都很值得我们推敲和学习。不过也可以看到,存取的操作不是特别的方便易用,需要我们自己去操作文件流。