Innovation and privacy go hand in hand here at Mozilla. To continue developing features and products that resonate with our users, we’re adopting a new a
You can technically modify any network weights however you want with whatever data you have lying around, but without the core training data you can’t verify that your modifications aren’t hurting the original capabilities. Fine-tuning (which LoRa is for) isn’t the same thing as modifying a trained network. You’re still generally stuck with their original trained capabilities you’re just reworking the final layer(s) to redirect/tune it towards your problem. You can’t add pet faces into a human face detector, and if a new technique comes out that could improve accuracy you can’t rebuild the model with it.
In any case, if the inference software is actually open source and all the necessary data is free of any intellectual property encumberances, it runs without internet access or non commodity hardware.
Then it’s open source enough to live in my browser.
So just free/noncorporate. A model is effectively a binary and the data is the source (the actual ML code is the compiler). If you don’t get the source, it’s not open source. A binary can be free and non-corporate, but it’s still not source code.
I mean, I would prefer a data set that’s properly open, “the pile” laion, open assistant and a pirate copy is every word, song, video ever written and spoken by man.
But for now I’d be happy to fully control my browser with an offline copy of mixtral or llama
You can technically modify any network weights however you want with whatever data you have lying around, but without the core training data you can’t verify that your modifications aren’t hurting the original capabilities. Fine-tuning (which LoRa is for) isn’t the same thing as modifying a trained network. You’re still generally stuck with their original trained capabilities you’re just reworking the final layer(s) to redirect/tune it towards your problem. You can’t add pet faces into a human face detector, and if a new technique comes out that could improve accuracy you can’t rebuild the model with it.
So just free/noncorporate. A model is effectively a binary and the data is the source (the actual ML code is the compiler). If you don’t get the source, it’s not open source. A binary can be free and non-corporate, but it’s still not source code.
I mean, I would prefer a data set that’s properly open, “the pile” laion, open assistant and a pirate copy is every word, song, video ever written and spoken by man.
But for now I’d be happy to fully control my browser with an offline copy of mixtral or llama