↧
MySQL 8.0.17: Connector/Python released (no replies)
↧
MySQL Connector/Python C Extension Prepared Statement Support (no replies)
↧
↧
8.0.17 bug: unknown collation utf8mb4_0900_ai_ci (1 reply)
Hi,
We are using Flask with SqlAlchemy, and mysql-connector-python is part of the dependency, we noticed the latest release rendered our service broken.
We did not specify the db connector version (a bad practice I know), so the recent deployment installed 8.0.17, then we tried to insert an emoji character into db which caused an error: sqlalchemy.exc.DatabaseError: (mysql.connector.errors.DatabaseError) 1273 (HY000): Unknown collation: 'utf8mb4_0900_ai_ci'.
As soon as we installed 8.0.16 manually, we are able to do so without a scratch.
So I was wodering if mysql-connector-python 8.0.17 is not respecting the fact that we have a 5.7.26 version mysql and use the default collation for mysql 8.0 to talk to mysql server?
Cheers.
Victor Z
We are using Flask with SqlAlchemy, and mysql-connector-python is part of the dependency, we noticed the latest release rendered our service broken.
We did not specify the db connector version (a bad practice I know), so the recent deployment installed 8.0.17, then we tried to insert an emoji character into db which caused an error: sqlalchemy.exc.DatabaseError: (mysql.connector.errors.DatabaseError) 1273 (HY000): Unknown collation: 'utf8mb4_0900_ai_ci'.
As soon as we installed 8.0.16 manually, we are able to do so without a scratch.
So I was wodering if mysql-connector-python 8.0.17 is not respecting the fact that we have a 5.7.26 version mysql and use the default collation for mysql 8.0 to talk to mysql server?
Cheers.
Victor Z
↧
Bug? InterfaceError: Failed getting warnings when executing a query with multiple statements with get_warnings (4 replies)
I am getting InterfaceError when I enable get_warnings and execute a query with multiple statements. I can't isolate the problem to an error in my own code so suspect a bug in the connector.
Connector version is mysql-connector-python-8.0.17 but 8.0.16 has the same issue.
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] on win32
MySQL 5.7
To demonstrate the problem:
import mysql.connector
cnx = mysql.connector.connect(host='xx',
user='xx',
password='xx',
database='xx',
use_pure=False,
get_warnings=True)
# First test, remove later
cur = cnx.cursor()
cur.execute('SELECT "a"+1')
for row in cur:
print(row)
print(cur.fetchwarnings())
cur.close()
# End first test
# Second test
cur = cnx.cursor()
for rs in cur.execute('SELECT "a"+1; SELECT 2', multi=True):
for row in rs:
print(row)
print(rs.fetchwarnings())
For the first execute(), we execute a single statement, iterate over the cursor, fetch and print warnings. Output as expected:
(1.0,)
[('Warning', 1292, "Truncated incorrect DOUBLE value: 'a'")]
For the second test, (you can remove the first test altogether), it will execute print(row) once, then an Exception happens. Output:
(1.0,)
Traceback (most recent call last):
File "C:\Program Files\Python37\lib\site-packages\mysql\connector\connection_cext.py", line 472, in cmd_query
raw_as_string=raw_as_string)
_mysql_connector.MySQLInterfaceError: Commands out of sync; you can't run this command now
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files\Python37\lib\site-packages\mysql\connector\cursor_cext.py", line 138, in _fetch_warnings
_ = self._cnx.cmd_query("SHOW WARNINGS")
File "C:\Program Files\Python37\lib\site-packages\mysql\connector\connection_cext.py", line 475, in cmd_query
sqlstate=exc.sqlstate)
mysql.connector.errors.DatabaseError: 2014 (HY000): Commands out of sync; you can't run this command now
During handling of the above exception, another exception occurred:
....etc....
Other notes:
* If you set get_warnings to False, no error happens and fetchwarnings() returns None
* If you remove the problem from the SQL code, no error happens and fetchwarnings() returns None
* use_pure can be True or False, the only difference is a slightly different traceback
* Using fetchall() instead of for row in rs gives the same result
* Many other variations give the same error.
Did anyone encounter the same problem? How did you solve it?
What am I doing wrong?
Connector version is mysql-connector-python-8.0.17 but 8.0.16 has the same issue.
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] on win32
MySQL 5.7
To demonstrate the problem:
import mysql.connector
cnx = mysql.connector.connect(host='xx',
user='xx',
password='xx',
database='xx',
use_pure=False,
get_warnings=True)
# First test, remove later
cur = cnx.cursor()
cur.execute('SELECT "a"+1')
for row in cur:
print(row)
print(cur.fetchwarnings())
cur.close()
# End first test
# Second test
cur = cnx.cursor()
for rs in cur.execute('SELECT "a"+1; SELECT 2', multi=True):
for row in rs:
print(row)
print(rs.fetchwarnings())
For the first execute(), we execute a single statement, iterate over the cursor, fetch and print warnings. Output as expected:
(1.0,)
[('Warning', 1292, "Truncated incorrect DOUBLE value: 'a'")]
For the second test, (you can remove the first test altogether), it will execute print(row) once, then an Exception happens. Output:
(1.0,)
Traceback (most recent call last):
File "C:\Program Files\Python37\lib\site-packages\mysql\connector\connection_cext.py", line 472, in cmd_query
raw_as_string=raw_as_string)
_mysql_connector.MySQLInterfaceError: Commands out of sync; you can't run this command now
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files\Python37\lib\site-packages\mysql\connector\cursor_cext.py", line 138, in _fetch_warnings
_ = self._cnx.cmd_query("SHOW WARNINGS")
File "C:\Program Files\Python37\lib\site-packages\mysql\connector\connection_cext.py", line 475, in cmd_query
sqlstate=exc.sqlstate)
mysql.connector.errors.DatabaseError: 2014 (HY000): Commands out of sync; you can't run this command now
During handling of the above exception, another exception occurred:
....etc....
Other notes:
* If you set get_warnings to False, no error happens and fetchwarnings() returns None
* If you remove the problem from the SQL code, no error happens and fetchwarnings() returns None
* use_pure can be True or False, the only difference is a slightly different traceback
* Using fetchall() instead of for row in rs gives the same result
* Many other variations give the same error.
Did anyone encounter the same problem? How did you solve it?
What am I doing wrong?
↧
python empty result set (1 reply)
I am trying to retrieve a result set from my database and no matter how I connect and execute the simple select statement, the results object always come back as None. I am showing the connection.is_connected() as True and the user I am using has full privileges on the database I'm accessing. Yes, there ARE rows in the table, about 500.
Thanks for any help,
Leif
Thanks for any help,
Leif
↧
↧
fetchmany bug? (1 reply)
Hello!
I have a test table with 90 061 rows and I want to fetch rows in batches, so I want to use fetchmany:
cursor = cnx.cursor()
query = "select id from test_table"
cursor.execute(query)
for i in range(105):
rows = cursor.fetchmany(1000)
print("rows returned : {}".format(len(rows)))
if not rows:
break;
Using connector versions 8.0.12 and above, the code do not stops when there is no more rows in cursor, fetchmany returns continously the last row from the set:
(...)
rows returned : 1000
rows returned : 1000
rows returned : 1000
rows returned : 1000
rows returned : 1000
rows returned : 61
rows returned : 1
rows returned : 1
rows returned : 1
rows returned : 1
rows returned : 1
rows returned : 1
Using the same code with connector v 8.0.11 gets me expected result:
(...)
rows returned : 1000
rows returned : 1000
rows returned : 1000
rows returned : 1000
rows returned : 1000
rows returned : 1000
rows returned : 1000
rows returned : 61
rows returned : 0
Python 3.6, MySQL 5.7
Is this a bug?
Best
Marcin
I have a test table with 90 061 rows and I want to fetch rows in batches, so I want to use fetchmany:
cursor = cnx.cursor()
query = "select id from test_table"
cursor.execute(query)
for i in range(105):
rows = cursor.fetchmany(1000)
print("rows returned : {}".format(len(rows)))
if not rows:
break;
Using connector versions 8.0.12 and above, the code do not stops when there is no more rows in cursor, fetchmany returns continously the last row from the set:
(...)
rows returned : 1000
rows returned : 1000
rows returned : 1000
rows returned : 1000
rows returned : 1000
rows returned : 61
rows returned : 1
rows returned : 1
rows returned : 1
rows returned : 1
rows returned : 1
rows returned : 1
Using the same code with connector v 8.0.11 gets me expected result:
(...)
rows returned : 1000
rows returned : 1000
rows returned : 1000
rows returned : 1000
rows returned : 1000
rows returned : 1000
rows returned : 1000
rows returned : 61
rows returned : 0
Python 3.6, MySQL 5.7
Is this a bug?
Best
Marcin
↧
quiz results in MYSql (no replies)
Hi,
We are working on a website and collecting quiz results. We are using Wordpress and ISpring Quizmaker. The results of the quiz are being pushed into the MYSql. We are trying to extract this is for analytical purposes.
The format looks a bit like the dictionary format of Python but I am not able to import it into Pandas.
Please, could you help me?
Thanks,
Naveen
We are working on a website and collecting quiz results. We are using Wordpress and ISpring Quizmaker. The results of the quiz are being pushed into the MYSql. We are trying to extract this is for analytical purposes.
The format looks a bit like the dictionary format of Python but I am not able to import it into Pandas.
Please, could you help me?
Thanks,
Naveen
↧
What is the 32 connection pooling limit for? (no replies)
Hi there
I am a Python Developer who has various hobbies in coding and one of them is coding a python-based automoderator bot on the Discord platform
However, due to its scale, I frequently run into issues with connection pooling due to the multiple concurrent requests to the pool system I am using, thus my bot shuts down due to using up all 32 connections (alongside message activity from Discord, there are tasks which query the DB as well).
Investigation of the code revealed that there is a hard-coded limit of 32 connections for a pool. Can anyone explain why that is there? I could not find anything that appeared to explain what this cap is for.
I edited my local copy of the module files to double this hard cap and started my bot with 40 connections and have not run into any issues, but I gather it's unsupported, but it still begs the question what this 32 limit is for.
Thanks for your replies in advance.
I am a Python Developer who has various hobbies in coding and one of them is coding a python-based automoderator bot on the Discord platform
However, due to its scale, I frequently run into issues with connection pooling due to the multiple concurrent requests to the pool system I am using, thus my bot shuts down due to using up all 32 connections (alongside message activity from Discord, there are tasks which query the DB as well).
Investigation of the code revealed that there is a hard-coded limit of 32 connections for a pool. Can anyone explain why that is there? I could not find anything that appeared to explain what this cap is for.
I edited my local copy of the module files to double this hard cap and started my bot with 40 connections and have not run into any issues, but I gather it's unsupported, but it still begs the question what this 32 limit is for.
Thanks for your replies in advance.
↧
How do I use a variable from a for loop in my sql query (no replies)
Hello there,
I am making a python script that connects to my mysql database and selects the name corresponding to the first id from the table.
It will then do an image search and download an image to which I can later add a path in my database.
This loops until it has done every id so every row has it's own unique image.
The script works already, however, I am having trouble with taking the id from python and putting it in the sql query.
Is there a way I can sort of close out of the query, enter the id variable, and resume the sql query with mysql.connector?
This is what it looks like right now:
for i in range(1, 2642):
habe = mysql.connector.connect(
host="localhost",
user="root",
passwd="",
database="habe"
)
id = i
mycursor = habe.cursor()
mycursor.execute("SELECT name FROM processor WHERE id = %s")
myresult = mycursor.fetchone()
i += 1
As I was looking this up, I saw people were recommending using %s as a placeholder but this doesn't work for me.
When I do this, it tells me I have an error in my sql syntax.
I am making a python script that connects to my mysql database and selects the name corresponding to the first id from the table.
It will then do an image search and download an image to which I can later add a path in my database.
This loops until it has done every id so every row has it's own unique image.
The script works already, however, I am having trouble with taking the id from python and putting it in the sql query.
Is there a way I can sort of close out of the query, enter the id variable, and resume the sql query with mysql.connector?
This is what it looks like right now:
for i in range(1, 2642):
habe = mysql.connector.connect(
host="localhost",
user="root",
passwd="",
database="habe"
)
id = i
mycursor = habe.cursor()
mycursor.execute("SELECT name FROM processor WHERE id = %s")
myresult = mycursor.fetchone()
i += 1
As I was looking this up, I saw people were recommending using %s as a placeholder but this doesn't work for me.
When I do this, it tells me I have an error in my sql syntax.
↧
↧
Please upgrade dependency protobuf to 3.11 (no replies)
Hi,
The latest Connector/Python released got dependency protobuf==3.6.1 which is released on Jul 28, 2018.
My team is using protobuf==3.11.0 . It's fine.
Is there a plan to upgrade dep protobuf to the 3.11?
Thank you.
The latest Connector/Python released got dependency protobuf==3.6.1 which is released on Jul 28, 2018.
My team is using protobuf==3.11.0 . It's fine.
Is there a plan to upgrade dep protobuf to the 3.11?
Thank you.
↧
mysql-connector-python 8.0.19 not compatible with Django 3.x (2 replies)
Django 3 removed six from django.utils. Latest version 8.0.19 is using it. You can simply fix it by replacing all the imports for six.
I have created a repository with a compatibility-django-3 branch in github fixing that issue regarding Django 3.
https://github.com/javiermatos/mysql-connector-python/tree/compatibility-django-3
Could you please integrate those changes? Also, I removed unused imports in code.
I have created a repository with a compatibility-django-3 branch in github fixing that issue regarding Django 3.
https://github.com/javiermatos/mysql-connector-python/tree/compatibility-django-3
Could you please integrate those changes? Also, I removed unused imports in code.
↧
MariaDB: MySQL Connector/Python update fails for some string lengths (no replies)
When updating a table entry, most strings work just fine but some don't. I assume it is related to the string length. For example, the strings "12345678" and "1234567890" work while the string "123456789" doesn't. Instead of correctly updating the entry, it throws a
OperationalError: 2055: Lost connection to MySQL server at '...:3306', system error: 60 Operation timed out
My script uses:
Python 3.7.6
mysql-connector-python 8.0.19
The db is running on the following system:
Server: Localhost via UNIX socket
Server type: MariaDB
Server connection: SSL is not being used Documentation
Server version: 10.4.12-MariaDB-1:10.4.12+maria~bionic - mariadb.org binary distribution
Protocol version: 10
Server charset: UTF-8 Unicode (utf8mb4)
Apache
Database client version: libmysql - mysqlnd 5.0.12-dev
Code that does not work:
mydb = mysql.connector.connect(...)
cursor = mydb.cursor()
cursor.execute('UPDATE table_1 SET name = "123456789" WHERE id = 1;')
OperationalError: 2055: Lost connection to MySQL server at '...:3306', system error: 60 Operation timed out
My script uses:
Python 3.7.6
mysql-connector-python 8.0.19
The db is running on the following system:
Server: Localhost via UNIX socket
Server type: MariaDB
Server connection: SSL is not being used Documentation
Server version: 10.4.12-MariaDB-1:10.4.12+maria~bionic - mariadb.org binary distribution
Protocol version: 10
Server charset: UTF-8 Unicode (utf8mb4)
Apache
Database client version: libmysql - mysqlnd 5.0.12-dev
Code that does not work:
mydb = mysql.connector.connect(...)
cursor = mydb.cursor()
cursor.execute('UPDATE table_1 SET name = "123456789" WHERE id = 1;')
↧
Python flask: mysql query cursor.execute(“SELECT * FROM tasksdb WHERE (id=%s)”, (id,)) returns () (no replies)
I have set up a SQL database in a Docker container and access it with a Flask program. In my test file, I want to return the item from the database depending on its id.
However, even though the task with the specified id exists in the database, I get () as result from any of these queries:
cursor.execute("SELECT * FROM tasksdb WHERE id='%s'", (id,))
cursor.execute("SELECT * FROM tasksdb WHERE id='%s'" % (id,))
cursor.execute('SELECT * FROM tasksdb WHERE id=%s', [id])
This is my full code:
from flask import Flask, request, Response
from collections import OrderedDict
import json
import MySQLdb
import json
app = Flask(__name__)
cursor = None
def get_db_connection():
global cursor
#only executed if cursor has not been initialized so far
if not cursor:
#db = MySQLdb.connect("some-mysql", "root", "DockerPasswort!", "demo")
#db = MySQLdb.connect("localhost:3306", "root", "DockerPasswort!", "demo")
db = MySQLdb.connect("127.0.0.1", "root", "DockerPasswort!", "demo", port=3306)
cursor = db.cursor()
return cursor
# Create a new task
@app.route('/v1/tasks', methods=['POST'])
def post():
cursor = get_db_connection()
data = request.get_json()
if "title" not in data:
return bulkadd(data)
is_completed=False
if "is_completed" in data:
is_completed=data["is_completed"]
notify=""
if "notify" in data:
notify=data["notify"]
task = data["title"]
sql='INSERT INTO tasksdb (task, is_completed, notify) VALUES (%s, %s, %s)'
cursor.execute(sql, [task, is_completed, notify])
cursor.execute('SELECT MAX(id) as maxid FROM tasksdb')
id=int(cursor.fetchall()[0][0])
return json.dumps({"id": id}), 201
#List all tasks created
@app.route('/v1/tasks', methods=['GET'])
def getall():
cursor = get_db_connection()
cursor.execute("SELECT * from tasksdb")
data = cursor.fetchall()
response_msg = list()
for row in data:
response_msg_link = OrderedDict()
response_msg_link["id"] = row[0]
response_msg_link["title"] = row[1]
is_completed=row[2]
if(is_completed==0):
is_completed=False
if(is_completed==1):
is_completed=True
response_msg_link["is_completed"] = is_completed
response_msg_link["notify"] = row[3]
response_msg.append(response_msg_link)
return json.dumps({"tasks": response_msg}), 200
#Get a specific task
@app.route('/v1/tasks/<id>', methods=['GET'])
def getone(id):
# I set up id to 22, because I am certain that this exists in the database:
id=int(22)
#cursor.execute("SELECT * FROM tasksdb WHERE id='%s'", (id,))
#cursor.execute("SELECT * FROM tasksdb WHERE id='%s'" % (id,))
cursor.execute('SELECT * FROM tasksdb WHERE id=%s', [id])
row = cursor.fetchall()
print(cursor.fetchall())
try:
response_msg_link = OrderedDict()
response_msg_link["id"] = row[0]
response_msg_link["title"] = row[1]
is_completed=row[2]
if(is_completed==0):
is_completed=False
if(is_completed==1):
is_completed=True
response_msg_link["is_completed"] = is_completed
response_msg_link["notify"] = row[3]
return json.dumps({response_msg}), 200
except:
return json.dumps({"error": "There is no task at that id"}), 404
Even though I have added many more entries in the database with the Flask app, this is the only result from the database (I created this directly in the sql bash shell):
SELECT * FROM tasksdb;
+----+---------------+--------------+-----------------------------+
| id | task | is_completed | notify |
+----+---------------+--------------+-----------------------------+
| 22 | My First Task | 0 | test@berkeley.edu |
+----+---------------+--------------+-----------------------------+
1 row in set (0.00 sec)
Thanks for your help!
However, even though the task with the specified id exists in the database, I get () as result from any of these queries:
cursor.execute("SELECT * FROM tasksdb WHERE id='%s'", (id,))
cursor.execute("SELECT * FROM tasksdb WHERE id='%s'" % (id,))
cursor.execute('SELECT * FROM tasksdb WHERE id=%s', [id])
This is my full code:
from flask import Flask, request, Response
from collections import OrderedDict
import json
import MySQLdb
import json
app = Flask(__name__)
cursor = None
def get_db_connection():
global cursor
#only executed if cursor has not been initialized so far
if not cursor:
#db = MySQLdb.connect("some-mysql", "root", "DockerPasswort!", "demo")
#db = MySQLdb.connect("localhost:3306", "root", "DockerPasswort!", "demo")
db = MySQLdb.connect("127.0.0.1", "root", "DockerPasswort!", "demo", port=3306)
cursor = db.cursor()
return cursor
# Create a new task
@app.route('/v1/tasks', methods=['POST'])
def post():
cursor = get_db_connection()
data = request.get_json()
if "title" not in data:
return bulkadd(data)
is_completed=False
if "is_completed" in data:
is_completed=data["is_completed"]
notify=""
if "notify" in data:
notify=data["notify"]
task = data["title"]
sql='INSERT INTO tasksdb (task, is_completed, notify) VALUES (%s, %s, %s)'
cursor.execute(sql, [task, is_completed, notify])
cursor.execute('SELECT MAX(id) as maxid FROM tasksdb')
id=int(cursor.fetchall()[0][0])
return json.dumps({"id": id}), 201
#List all tasks created
@app.route('/v1/tasks', methods=['GET'])
def getall():
cursor = get_db_connection()
cursor.execute("SELECT * from tasksdb")
data = cursor.fetchall()
response_msg = list()
for row in data:
response_msg_link = OrderedDict()
response_msg_link["id"] = row[0]
response_msg_link["title"] = row[1]
is_completed=row[2]
if(is_completed==0):
is_completed=False
if(is_completed==1):
is_completed=True
response_msg_link["is_completed"] = is_completed
response_msg_link["notify"] = row[3]
response_msg.append(response_msg_link)
return json.dumps({"tasks": response_msg}), 200
#Get a specific task
@app.route('/v1/tasks/<id>', methods=['GET'])
def getone(id):
# I set up id to 22, because I am certain that this exists in the database:
id=int(22)
#cursor.execute("SELECT * FROM tasksdb WHERE id='%s'", (id,))
#cursor.execute("SELECT * FROM tasksdb WHERE id='%s'" % (id,))
cursor.execute('SELECT * FROM tasksdb WHERE id=%s', [id])
row = cursor.fetchall()
print(cursor.fetchall())
try:
response_msg_link = OrderedDict()
response_msg_link["id"] = row[0]
response_msg_link["title"] = row[1]
is_completed=row[2]
if(is_completed==0):
is_completed=False
if(is_completed==1):
is_completed=True
response_msg_link["is_completed"] = is_completed
response_msg_link["notify"] = row[3]
return json.dumps({response_msg}), 200
except:
return json.dumps({"error": "There is no task at that id"}), 404
Even though I have added many more entries in the database with the Flask app, this is the only result from the database (I created this directly in the sql bash shell):
SELECT * FROM tasksdb;
+----+---------------+--------------+-----------------------------+
| id | task | is_completed | notify |
+----+---------------+--------------+-----------------------------+
| 22 | My First Task | 0 | test@berkeley.edu |
+----+---------------+--------------+-----------------------------+
1 row in set (0.00 sec)
Thanks for your help!
↧
↧
Install pip command in Mint (no replies)
I have to install the Django package to my mint distribution. I want to accomplish this by using pip command but can not find the pip command. How can I install the pip command ? Thanks
↧
NoSQL Development in MySQL Using Python (no replies)
↧
check file Existing with Python Script (no replies)
Are there any way to check the file existence in Python script.
↧
Restore stored procedures via Python script (no replies)
Hello:
I want to restore a database from the sql script created by mysqldump.exe. Using mysql_connector, the cursor stops when it hits the section for the stored procedures. I read something about the DELIMITER command only working from the command line or Workbench.
Is there a way for a python script to perform a CREATE PROCEDURE script without having to just call mysql.exe to do a restore?
Cheers and thanks,
Simon Norton
I want to restore a database from the sql script created by mysqldump.exe. Using mysql_connector, the cursor stops when it hits the section for the stored procedures. I read something about the DELIMITER command only working from the command line or Workbench.
Is there a way for a python script to perform a CREATE PROCEDURE script without having to just call mysql.exe to do a restore?
Cheers and thanks,
Simon Norton
↧
↧
Check Python Interpreter Version (no replies)
Are there any way to check the Python interpreter version without entering the Python shell or writing Python script?
↧
MySQL Connector/Python 8.0.20 has been released (no replies)
↧
mysql-connector-python starting from 8.0.12 goes to infinite loop when iterates till the end of table (4 replies)
I have CentOS 7.8, Python 3.6.8 and Django 2.2.12 installed via pip.
When I install mysql-connector-python 8.0.6 or 8.0.5 (also via pip), everything works fine.
But for versions 8.0.12, 14, and 19, when I use Django queryset to iterate through all the rows of a simple table (with 20 fields and 200 rows), it goes to an infinite loop and takes all my RAM (about 2GB) and then crashes.
Here are examples how the stack trace can look after several seconds of executing (in 8.0.6 it takes much less than a second to finish this loop):
>>> for r in Club.objects.all():
... pass
...
^CTraceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 272, in __iter__
self._fetch_all()
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 1179, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 64, in __iter__
obj = model_cls.from_db(db, init_list, row[model_fields_start:model_fields_end])
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/base.py", line 507, in from_db
new = cls(*values)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/base.py", line 497, in __init__
post_init.send(sender=cls, instance=self)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/dispatch/dispatcher.py", line 178, in send
for receiver in self._live_receivers(sender)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/dispatch/dispatcher.py", line 260, in _live_receivers
receiver = receiver()
File "/usr/lib64/python3.6/weakref.py", line 67, in __call__
obj = super().__call__()
KeyboardInterrupt
^CTraceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 274, in __iter__
self._fetch_all()
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 1242, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 55, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1173, in execute_sql
return list(result)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1552, in cursor_iter
for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1552, in <lambda>
for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/utils.py", line 96, in inner
return func(*args, **kwargs)
File "/usr/local/venv/main/lib64/python3.6/site-packages/mysql/connector/cursor_cext.py", line 505, in fetchmany
if self._nextrow and self._nextrow[0]:
KeyboardInterrupt
^CTraceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 274, in __iter__
self._fetch_all()
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 1242, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 55, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1173, in execute_sql
return list(result)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1552, in cursor_iter
for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1552, in <lambda>
for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/backends/utils.py", line 22, in __getattr__
def __getattr__(self, attr):
KeyboardInterrupt
This problem appears only if I iterate to the end of the table. My table has ~300 rows, and there are no mistakes when iterating through Club.objects.all()[:100], but there is an infinite loop when iterating through any of:
Club.objects.all()
Club.objects.all()[:1000]
Club.objects.all()[5:]
I also tried 8.0.11 but I got a different error: 'datetime.date' object has no attribute 'split'. Versions 8.0.7-8.0.10 aren't available in pip.
Any ideas why this can happen?
When I install mysql-connector-python 8.0.6 or 8.0.5 (also via pip), everything works fine.
But for versions 8.0.12, 14, and 19, when I use Django queryset to iterate through all the rows of a simple table (with 20 fields and 200 rows), it goes to an infinite loop and takes all my RAM (about 2GB) and then crashes.
Here are examples how the stack trace can look after several seconds of executing (in 8.0.6 it takes much less than a second to finish this loop):
>>> for r in Club.objects.all():
... pass
...
^CTraceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 272, in __iter__
self._fetch_all()
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 1179, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 64, in __iter__
obj = model_cls.from_db(db, init_list, row[model_fields_start:model_fields_end])
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/base.py", line 507, in from_db
new = cls(*values)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/base.py", line 497, in __init__
post_init.send(sender=cls, instance=self)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/dispatch/dispatcher.py", line 178, in send
for receiver in self._live_receivers(sender)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/dispatch/dispatcher.py", line 260, in _live_receivers
receiver = receiver()
File "/usr/lib64/python3.6/weakref.py", line 67, in __call__
obj = super().__call__()
KeyboardInterrupt
^CTraceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 274, in __iter__
self._fetch_all()
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 1242, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 55, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1173, in execute_sql
return list(result)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1552, in cursor_iter
for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1552, in <lambda>
for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/utils.py", line 96, in inner
return func(*args, **kwargs)
File "/usr/local/venv/main/lib64/python3.6/site-packages/mysql/connector/cursor_cext.py", line 505, in fetchmany
if self._nextrow and self._nextrow[0]:
KeyboardInterrupt
^CTraceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 274, in __iter__
self._fetch_all()
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 1242, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 55, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1173, in execute_sql
return list(result)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1552, in cursor_iter
for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1552, in <lambda>
for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/backends/utils.py", line 22, in __getattr__
def __getattr__(self, attr):
KeyboardInterrupt
This problem appears only if I iterate to the end of the table. My table has ~300 rows, and there are no mistakes when iterating through Club.objects.all()[:100], but there is an infinite loop when iterating through any of:
Club.objects.all()
Club.objects.all()[:1000]
Club.objects.all()[5:]
I also tried 8.0.11 but I got a different error: 'datetime.date' object has no attribute 'split'. Versions 8.0.7-8.0.10 aren't available in pip.
Any ideas why this can happen?
↧