-
Notifications
You must be signed in to change notification settings - Fork 22
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ReadV2 in TsFile #144
Merged
Merged
ReadV2 in TsFile #144
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit
Hold shift + click to select a range
44d26bb
TMP save 01-10
2fee2f0
TMP save 01-16
9a2b0b0
add getSize() to TimeValuePair
e1dc833
modify interface of SeriesChunk
ef5b406
fix a bug in last commit
a9d19e3
add close() method to TimeValuePair interface
a66a86f
add single series reader for one TsFile
fd3fa8e
add constructor for TsFileIOWriter
liukun4515 2026d94
add Serializable to TimeValuePair class
c9e7bde
Merge branch 'f_zjr_read_rebuild_pr' of github.com:thulab/tsfile into…
2be7808
add startRowGroup and endRowGroup function for tsfile
liukun4515 720513c
add datatype to TsPrimitiveType
efd1fcf
Merge branch 'f_zjr_read_rebuild_pr' of github.com:thulab/tsfile into…
87ee4e7
remove unused lines
05393db
fix a bug in SeriesReaderFromSingleFile where the fileStream is not c…
59e453d
add getByType for TsPrimitiveType
999b00b
change log level from info to debug
8784e02
cancel fill tsfile
liukun4515 f2e7723
remove cache and query in seieswriteimpl
liukun4515 d0c1eb1
add clear() function to Cache
5f9f0bb
remove logback.xml
8a20462
add error log in flush timeseries data
liukun4515 bd77c43
add error in flush timeseries data
liukun4515 aff995d
remove device id from FileSchema
kr11 3b4399a
Remove unused import
kr11 ffe2bbd
remove the checkIsDeltaExist function, add checkIsTimeSeriesExist fun…
liukun4515 d39d885
remove the get memory function
liukun4515 2c4e109
add more constructor to SeriesReader
a079888
Merge branch 'f_zjr_read_rebuild_pr' of github.com:thulab/tsfile into…
dca0ac6
repair some codes according to comments from Pull Request
6a65f5c
modify the code according to the comments in pr
liukun4515 bdf10fe
modify PageReader
2b9e7cc
fix some codes according to comments
3e5b35e
Merge branch 'f_zjr_read_rebuild_pr' of github.com:thulab/tsfile into…
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
add error log in flush timeseries data
- Loading branch information
commit 8a20462665597bb92b3b40b0a9047db42a635420
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
10 changes: 10 additions & 0 deletions
10
src/main/java/cn/edu/tsinghua/tsfile/timeseries/readV2/datatype/Test.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
package cn.edu.tsinghua.tsfile.timeseries.readV2.datatype; | ||
|
||
public class Test { | ||
|
||
public static void main(String[] args) { | ||
// TODO Auto-generated method stub | ||
new TsPrimitiveType.TsBoolean(false); | ||
} | ||
|
||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -44,6 +44,9 @@ public void writePage(ListByteArrayOutputStream listByteArray, int valueCount, S | |
// compress the input data | ||
if (this.minTimestamp == -1) | ||
this.minTimestamp = minTimestamp; | ||
if(this.minTimestamp==-1){ | ||
LOG.error("Write page error, {}, minTime:{}, maxTime:{}",desc,minTimestamp,maxTimestamp); | ||
} | ||
this.maxTimestamp = maxTimestamp; | ||
int uncompressedSize = listByteArray.size(); | ||
ListByteArrayOutputStream compressedBytes = compressor.compress(listByteArray); | ||
|
@@ -76,22 +79,16 @@ public void writePage(ListByteArrayOutputStream listByteArray, int valueCount, S | |
estimateMaxPageMemSize()); | ||
} | ||
|
||
public Pair<List<ByteArrayInputStream>, CompressionTypeName> query() { | ||
|
||
|
||
List<ByteArrayInputStream> backupPageList = buf.transform(); | ||
Pair<List<ByteArrayInputStream>, CompressionTypeName> ret = new Pair<List<ByteArrayInputStream>, CompressionTypeName>( | ||
backupPageList, compressor.getCodecName()); | ||
return ret; | ||
} | ||
|
||
private void resetTimeStamp() { | ||
if (totalValueCount == 0) | ||
minTimestamp = -1; | ||
} | ||
|
||
@Override | ||
public void writeToFileWriter(TsFileIOWriter writer, Statistics<?> statistics) throws IOException { | ||
if(minTimestamp==-1){ | ||
LOG.error("Write page error, {}, minTime:{}, maxTime:{}",desc,minTimestamp,maxTimestamp); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 这里如果出错了,还继续执行下去? |
||
} | ||
writer.startSeries(desc, compressor.getCodecName(), desc.getType(), statistics, maxTimestamp, minTimestamp); | ||
long totalByteSize = writer.getPos(); | ||
writer.writeBytesToStream(buf); | ||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这里如果出错了,还继续执行下去?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
需要,这个是当时测试出来的错误,如果出问题想用error日志记录下来。