LDPC research

[sourcerer]

I did some LDPC research lately and developed an algorithm to extract the LDPC parameters from a donor or a patient controller and to fix the bit errors on patient dumps. I could need some help from people with a logic analyzer / VNR / FE / PC3K-Flash to test my solution.

[Arch Stanton]

While I try to catch my breath: can you explain how someone with FE could help?

[Amarbir[CDR-Labs]]

Well,
It was a pleasure speaking to you today ,I would be contributing a lot of cases and access to VNR/PC3K .

[Arch Stanton]

Well,
It was a pleasure speaking to you today, thanks for your explanation.

[csava]

Based on my personal experience, developers of 3K and VNR tend to be more reserved and less active within the community. Despite their strong capabilities, staying updated with the latest and comprehensive information can be challenging without community interaction.
Regarding LDPC, what are the specific tasks?

[Gregory]

I did some LDPC research lately and developed an algorithm to extract the LDPC parameters from a donor or a patient controller and to fix the bit errors on patient dumps. I could need some help from people with a logic analyzer / VNR / FE / PC3K-Flash to test my solution.

Is this yours?
https://futureware.at/CrowdFunding/flashextractor.php

This is related to your github?

If yes, why is this doing behind sergey back?

https://futureware.at/CrowdFunding/flashextractor.php

Thanks for answer.

[fzabkar]

I did some LDPC research lately and developed an algorithm to extract the LDPC parameters from a donor or a patient controller and to fix the bit errors on patient dumps. I could need some help from people with a logic analyzer / VNR / FE / PC3K-Flash to test my solution.

Is this yours?
https://futureware.at/CrowdFunding/flashextractor.php

This is related to your github?

If yes, why is this doing behind sergey back?

https://futureware.at/CrowdFunding/flashextractor.php

Thanks for answer.

What's your problem, Greg?

[sourcerer]

Hi,

Is this yours?
https://futureware.at/CrowdFunding/flashextractor.php

This is related to your github?

If yes, why is this doing behind sergey back?

https://futureware.at/CrowdFunding/flashextractor.php

Thanks for answer.[/quote]

Yes, that's mine. And it is related to my github, absolutely.
It's a draft for a CrowdFunding campaign that I am currently planning together with Sergey, and Sergey approved a slightly older version of it already, but we wanted to continue working and refining it before making it public. There are still a few details to be sorted out, but the big picture of the plan is there.

Basically, I heard that Sergey wanted to stop the development and the support of Flash Extractor, and I thought that this would a big loss for the data recovery industry, so I discussed it with Sergey and several other people, and proposed to help with the development and to make it opensource. Some feedback I got was that raising money for FE without having LDPC support would be futile in the long run, so I started to work on LDPC and developed a solution. The idea now is to add the LDPC solution into the offer, to also make the LDPC solution opensource and available to public when the campaign is successful. But the long-term goal is to preserve the investments Sergey and everyone else did into FE, to not just maintain it, but to make it far better, by working on it and with it together as a community!

So I would suggest the following: If anyone has any feedback on the campaign draft there, please let me know so that I can work it in. I will further improve it and we will likely launch the campaign in a few days. If anyone wants to join and help the effort, please let me know.

PS: Thanks to everyone who helped me so far, and thanks for pushing me to finally make this plan public, sometimes I overdo drafting

[Gregory]

What's your problem, Greg?

Wrong second link, I want ask for this http://flash-extractor.com/forum/viewtopic.php?t=12967

In Sergey opinion this is fraud, so i'm wondering what's going on here?

[Lardman]

LDPC development work is going to be needed but it doesn't appear to be coming from anywhere else at the moment. Anything is better than nothing, and opensource is even better, there are plenty who could assist that are not involved directly in DR so why waste that resource.

With hardware running out for FE that will become a closed environment entirely and we're left with a duopoly with pc3K and VNR that isn't going to drive development forward which is not good for any of us.

As to "fraud" I think Sergey was probably looking for word "scam" expressing his concerns the project will turn into vapourware from general complexity or from feature creep. Which having read the page, it does make a lot of promises. The LDPC trials will go some way to answer credibility and the future development side of things.

My personal opinion is that if you're going to work on more complex and current requirements then you need to have the correct foundations and building onto FE may not be the best place to start from, it is after all a legacy application. Perhaps there's some secret sauce in there that you can't recreate or you're trying to preserve the workflow for the end user?

[fzabkar]

What's your problem, Greg?

Wrong second link, I want ask for this http://flash-extractor.com/forum/viewtopic.php?t=12967

In Sergey opinion this is fraud, so i'm wondering what's going on here?

To me, the obvious answer is to provide @sourcerer with a raw NAND dump from a working drive and then see whether he can provide the files or file list. Of course, the controller should be one that @sourcerer chooses, as long as it is one which is regarded as unsolvable. In this way @sourcerer doesn't need to disclose his algorithm.

[Gregory]

@sourcerer With what page structure your LDPC algorithm was testing: 4576, 4568_4632, 4628_4692, 4564_4644, 2280, 2324?

[sourcerer]

At the moment I am simulating it with a simple 4000 byte page structure which contains 1024-DATA 476-ECC 1024-DATA 476-ECC 8-SA. My simulator can easily be parameterized for different geometries. Can you provide a few blocks of dump and all the details about the geometry of the case you have?

[Gregory]

Can you provide a few blocks of dump and all the details about the geometry of the case you have?

I have access to few donors, if I find lite time read few blocks with 0x77 pattern.

btw. 1024bytes protect by 476bytes ?? for example NAND with LDPC have 4096byte data, protect by 472 bytes ecc - 4 times more data.
Do your algo use only hard decision decoding technique or also soft decision?

[sourcerer]

Thank you, I will test it with 4096 and 472 too. From my point of view, choosing a good LDPC length not just depends on the size of the data area, but especially on the number of bit-errors that are to be expected. The more noise you have there, the larger ECC area I would suggest to use. I have experienced quite noisy and quite stable NAND flash in the past. The minimum sum algorithm I am using at the moment is using only hard decisions as far as I know, but from what I have seen we should be able to use various different decoders, both hard-decision and soft-decision. I think it would make sense to combine soft-decision decoders with Read-Retry. Not all decoders will be suitable for us, but from the decoder market overview I have at the moment I think that we will find about 10 decoders that we can use, and I hope that one of them will be fast enough for our needs.

[csava]

I believe that the feasibility of LDPC soft decision at the current stage is relatively low, as it requires obtaining accurate soft information from NAND chips. If the acquired soft information is incorrect, it could lead to catastrophic consequences. Currently, almost all developers cannot guarantee that their utilized ReadRetry implementations are fully compliant with the specifications, as those instructions are often derived from reverse engineering. Perhaps certain values do not belong to the addresses of those registers. However, we have no way of knowing the potential repercussions if those erroneous values are used; they could potentially disrupt the entire dump.

[sourcerer]

I am happy to present you my solution for LDPC now:
The necessary tools are available on https://github.com/thesourcerer/drresearch/
and I made a first video that shows how to use it on https://www.youtube.com/watch?v=z93vdLnfoDs

[fzabkar]

The necessary tools are available on https://github.com/thesourcerer/drresearch/

404 - "This is not the web page you are looking for"

[sourcerer]

Sorry, the is a typo, the correct link is https://github.com/thesourcerer8/drresearch/

[arvika]

How you plan to find read parity check matrix? So far we can see patter written on video.