My critique on DALL.E (The new text to image model...
# thinking-together
s
My critique on DALL.E (The new text to image models from OpenAI) - https://www.linkedin.com/posts/shubhadeep-roychowdhury_right-openai-dalle-happened-my-whole-social-activity-6752552570746744832-vnIJ Any thoughts are welcome!
👍 4
s
Solid critique. I'm not sure if all the criticism of these large transformer models being basically impossible to train will stop them from happening because now it could be a huge competitive advantage for entrenched tech companies
especially now that selling cloud services is all the rage. There is big economic incentive to have powerful models hidden behind cloud services and sold
It would be nice if they published full papers + released source + documented the hardware required to train these models
so even if it cost like ~$5M to train one of these, theoretically it could be reproduced
even for future generations that might have similar compute in a more accessible factor
w
Will the price go down though?
s
It is already happening. OpenAI never published their model of GPT-3 and instead selling an "API" access to people. Watson, AzureAI etc had been doing that for long. We should expect companies like DeepMind follow suit as well. Here is the funny thing(according to me), it is a (somewhat) vicious circle. Big Company with money -> Can train big model -> Can put hundreds of dollars in compute and marketing alike -> Can bring media, money, attention -> Bigger company -> Can train bigger model... Rinse and Repeat. At least that is how I see it. And for that really important topics, such as Neuro Symbolic (or otherwise) models which can become really data efficient, Ethics in AI, Explainability and Interpretability, Alternative computing models for intelligence (apart form RL) etc. are never really surfacing. The reason being the circle I mentioned above.
💯 1
1
👍 2
Also let's. not forget wordseye (https://www.wordseye.com/) which does the same (often much better) and predates DL.
👀 1
m
with regard to point #3, you realize all the permutations are pre-cached right? that's why it's not free form - it's too expensive to give an open endpoint to anyone.