Quantcast
Channel: MySQL Forums - Connector/Python
Viewing latest articles
Browse all 384 Browse latest View live
↧

pooled connection trauma (6 replies)

July 28, 2022, 1:21 am
$
0
0
I hit an issue caused by the need to move to connection pools.
I was initially using Connector/Python 8.0.21 with mySQL server 5.6.10 however discovered that an exception would be thrown if I created the pool with pool_reset_session=True, when closing the connection, mysql.connector.errors.NotSupportedError: MySQL version 5.7.2 and earlier does not support COM_RESET_CONNECTION.
I was surprised by this as release notes say that this issue was fixed in Connector/Python 8.0.16

In moving to Connector/Python 8.0.29 the exception thrown, mentioned above, was fixed.
However I then found that my stored procedures would cause the following exception: mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax

Currently the only way around this replacement of stored procedures with normal queries.

Any thoughts on what I have described appreciated.
↧
Search

'Not naive datetime (tzinfo is already set)') (no replies)

August 9, 2022, 4:46 am
$
0
0
Hi Guys,

Anyone seen this message in SQL operations?
(venv) root@84ff53e702e9:/www_root# python manage.py migrate
Traceback (most recent call last):
File "manage.py", line 24, in <module>
main()
File "manage.py", line 20, in main
execute_from_command_line(sys.argv)
File "/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
utility.execute()
File "/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/venv/lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv
self.execute(*args, **cmd_options)
File "/venv/lib/python3.8/site-packages/django/core/management/base.py", line 398, in execute
output = self.handle(*args, **options)
File "/venv/lib/python3.8/site-packages/django/core/management/base.py", line 89, in wrapped
res = handle_func(*args, **kwargs)
File "/venv/lib/python3.8/site-packages/django/core/management/commands/migrate.py", line 92, in handle
executor = MigrationExecutor(connection, self.migration_progress_callback)
File "/venv/lib/python3.8/site-packages/django/db/migrations/executor.py", line 18, in __init__
self.loader = MigrationLoader(self.connection)
File "/venv/lib/python3.8/site-packages/django/db/migrations/loader.py", line 53, in __init__
self.build_graph()
File "/venv/lib/python3.8/site-packages/django/db/migrations/loader.py", line 220, in build_graph
self.applied_migrations = recorder.applied_migrations()
File "/venv/lib/python3.8/site-packages/django/db/migrations/recorder.py", line 78, in applied_migrations
return {(migration.app, migration.name): migration for migration in self.migration_qs}
File "/venv/lib/python3.8/site-packages/django/db/models/query.py", line 280, in __iter__
self._fetch_all()
File "/venv/lib/python3.8/site-packages/django/db/models/query.py", line 1324, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/venv/lib/python3.8/site-packages/django/db/models/query.py", line 68, in __iter__
for row in compiler.results_iter(results):
File "/venv/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1122, in apply_converters
value = converter(value, expression, connection)
File "/venv/lib/python3.8/site-packages/django/db/backends/mysql/operations.py", line 313, in convert_datetimefield_value
value = timezone.make_aware(value, self.connection.timezone)
File "/venv/lib/python3.8/site-packages/django/utils/timezone.py", line 242, in make_aware
return timezone.localize(value, is_dst=is_dst)
File "/venv/lib/python3.8/site-packages/pytz/__init__.py", line 245, in localize
raise ValueError('Not naive datetime (tzinfo is already set)')

Regards
↧
↧

python mysql client sample connection using kerberos (on windows) (no replies)

August 9, 2022, 7:47 pm
$
0
0
I'm having trouble finding any samples on connecting to a mysql db using the python client with sso (kerberos) from windows. Is there any chance documentation on this front can be added?
↧

SSL routines:ssl_choose_client_version:unsupported protocol (no replies)

August 16, 2022, 1:04 am
$
0
0
I have mysql-connector-python==8.0.29 installed on a windows desktop machine (python 3.7) connecting to an mysql server (mySQL server 5.6.10 on an EC2 instance) and connection pools are created and used just fine.

I then started to test the use of connection pools, on the target system, where the script that creates them runs (is the same as the windows machine above) but run on another AWS EC2 instance (Ubuntu 20.04.3 LTS (GNU/Linux 5.11.0-1022-aws x86_64)) and connects to the same mySQL server as above.

The AWS instance running the script is using mysql-connector-python==8.0.29 installed, however is running Python 3.8 not 3.7 (as with the windows machine). However I don't think the python version is causing the issue.

I get the following issue, which is potentially nothing to-do with mySQL and everything todo with SSL!

However I wondered if anyone had any recommendations/thoughts/wisdom?

-- Logs begin at Thu 2022-08-04 15:53:25 UTC. --
Aug 16 07:31:50 ip-address python3[126843]: return CMySQLConnection(*args, **kwargs)
Aug 16 07:31:50 ip-address python3[126843]: File "/usr/local/lib/python3.8/dist-packages/mysql/connector/connection_cext.py", line 94, in __init__
Aug 16 07:31:50 ip-address python3[126843]: self.connect(**kwargs)
Aug 16 07:31:50 ip-address python3[126843]: File "/usr/local/lib/python3.8/dist-packages/mysql/connector/abstracts.py", line 1052, in connect
Aug 16 07:31:50 ip-address python3[126843]: self._open_connection()
Aug 16 07:31:50 ip-address python3[126843]: File "/usr/local/lib/python3.8/dist-packages/mysql/connector/connection_cext.py", line 251, in _open_connection
Aug 16 07:31:50 ip-address python3[126843]: raise errors.get_mysql_exception(msg=exc.msg, errno=exc.errno,
Aug 16 07:31:50 ip-address python3[126843]: mysql.connector.errors.InterfaceError: 2026 (HY000): SSL connection error: error:1425F102:SSL routines:ssl_choose_client_version:unsupported protocol
Aug 16 07:31:50 ip-address systemd[1]: run_production_tests.service: Main process exited, code=exited, status=1/FAILURE
Aug 16 07:31:50 ip-address systemd[1]: run_production_tests.service: Failed with result 'exit-code'.
↧

Python mysql-connector doesn't handle numpy.nan (1 reply)

August 17, 2022, 8:06 am
$
0
0
Hi,

My workflow requires that I bulk insert pandas dataframes to a mysql database.

pandas uses numpy.nan to represent null values in a dataframe's float columns.
Unfortunately, when converting said values to a query, the mysql-connector converts numpy.nan to a string; 'nan', failing to insert it to the float column in the db.

I have a code workaround where I replace all numpy.nan values with None, but that is compute intensive and takes up much more space.

Is it possible to open a pull request on mysql-connector github to fix this?
↧
↧

OverflowError: Python int too large to convert to C long (2 replies)

October 8, 2022, 2:16 am
$
0
0
Hi,
Can someone help me with this problem?

The code below I ran on python 3.9.7 with mysql-connector-python 8.0.23 has no any problem.

#########################################################################
import mysqlx

ss = mysqlx.get_session({'host':'localhost', 'port':33060, 'user':'root', 'password':'xxxx'})
ss.drop_schema('testx')
ss.create_schema('testx')

db = ss.get_schema('testx')
db.create_collection('test')

col = db.get_collection('test')
col.add({'_id':'test', 'item':2147483648}).execute() # 8.0.23 pass, but 8.0.24 ~ 8.0.30 fail
#col.add({'_id':'test', 'item':2147483647}).execute() # this line is OK for any version

ss.drop_schema('testx')
ss.close()
#########################################################################

But, after I upgrading mysql-connector-python to 8.0.24 ~ 8.0.30(python version is 3.10.7),
it runs fail with the following error message.

-------------------------------------------------------------------------
OverflowError: Python int too large to convert to C long

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "R:\test.py", line 12, in <module>
col.add({'_id':'test', 'item':2147483648}).execute() # 8.0.23 pass, but 8.0.30 fail
File "C:\Users\zmsun\AppData\Local\Programs\Python\Python310\lib\site-packages\mysqlx\statement.py", line 711, in execute
return self._connection.send_insert(self)
File "C:\Users\zmsun\AppData\Local\Programs\Python\Python310\lib\site-packages\mysqlx\connection.py", line 494, in wrapper
result = func(self, *args, **kwargs)
File "C:\Users\zmsun\AppData\Local\Programs\Python\Python310\lib\site-packages\mysqlx\connection.py", line 1150, in send_insert
self.protocol.send_msg(msg_type, msg)
File "C:\Users\zmsun\AppData\Local\Programs\Python\Python310\lib\site-packages\mysqlx\protocol.py", line 835, in send_msg
self._writer.write_message(mysqlxpb_enum(msg_type), msg)
File "C:\Users\zmsun\AppData\Local\Programs\Python\Python310\lib\site-packages\mysqlx\protocol.py", line 268, in write_message
msg_size = msg.byte_size(msg)
File "C:\Users\zmsun\AppData\Local\Programs\Python\Python310\lib\site-packages\mysqlx\protobuf\__init__.py", line 541, in byte_size
else len(encode_to_bytes(msg.serialize_to_string()))
File "C:\Users\zmsun\AppData\Local\Programs\Python\Python310\lib\site-packages\mysqlx\protobuf\__init__.py", line 489, in serialize_to_string
return Protobuf.mysqlxpb.serialize_message(self._msg)
SystemError: <built-in function serialize_message> returned a result with an exception set
-------------------------------------------------------------------------

Is there anything I can change to fix this? Thanks.
↧

Problem with inserting data into MySQL (no replies)

October 29, 2022, 1:18 am
$
0
0
Hi
I want to read the data of a sensor and transfer it to MySQL local host with "mysql.connector". The problem is that it reads the data as a list containing 500 data (that is, it reads 500 to 500, not one by one). This is my code:

master_data = master_task.read(number_of_samples_per_channel=500)
sql = "INSERT INTO vib_data2 (data) VALUES (%s)"
val = list(master_data, )
mycursor.executemany(sql, val)
mydb.commit()


I got this error:
"Could not process parameters: float(-0.15846696725827258), it must be of type list, tuple or dict"


I can fix the problem with this code:

for i in master_data:
sql = "INSERT INTO vib_data2 (data) VALUES (%s)"
val = (str(I),)
mycursor.execute(sql, val)
mydb.commit()


But the code execution time is very long (more than one second) and thus some of the data is lost.
Thank you for helping to correct the code
↧

I cannot implement python along with MySQL integration (2 replies)

September 9, 2023, 4:30 pm
$
0
0
I am facing issue that I can not implement MySQL in my system what would be main reason
↧
Search

mysql.connector.errors.InterfaceError: 2003 (2 replies)

April 5, 2023, 1:51 am
↧
↧

Upgrade to a newer protobuf version (3 replies)

April 21, 2023, 6:54 am
$
0
0
In Apache Airflow we have mysql-conneector-python integration that currently holds us back from upgrading to protobuf 4.

Having limitation to Requires-Dist: protobuf (<Requires-Dist: protobuf (<=3.20.3,>=3.11.0) prevents us from upgrading most of other providers - see apache/airflow#30067, Until mysql-connector-python fixes the dependency limitation, we will be forced to stop releasing the provider together with the others (it also means it will not be compatible with the other providers and newer versions of airflow when released

Can you please let us know if there are plans to upgrade the mysql-connector-python python sdk to not limit the protobuf? Otherwise we will have to exclude the mysql provider and stop releasing it.

Also see the devlist discussion thread I started about it (based on yandexcloud provider, but it applies to mysql too).

Thanks in advance!
↧

Connection Arguments - host (2 replies)

March 27, 2023, 6:59 am
$
0
0
Hi

How to configure host argument correctly when I have SQL DB on server instance?
I have configured user only for this instance.

When im trying put:
host='server_name\instance_name'
I have an error:
2005 (HY000): Unknown MySQL server host 'server_name\instance_name' (11001)

When im trying put only:
host='server_name'
I have only the information:
2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0

Regards
Mariusz
↧

Support for protobufs 4? (3 replies)

August 17, 2023, 5:29 am
$
0
0
The current latest version of mysql-connector-python (8.0.32) has a dependency conflict with the oldest version of google-ads (19.0.0) that works with current Google APIs (v12+).

The conflict is caused by:
google-ads 19.0.0 depends on protobuf>=4.21.5
mysql-connector-python 8.0.32 depends on protobuf<=3.20.3 and >=3.11.0


Are there plans to switch to protobuf 4.21.5+ under the covers?
↧

Not all records - EOF deprecation / protocol.py (1 reply)

April 21, 2023, 9:15 am
$
0
0
On larger queries - not all of the records are making it to my .fetchall() statement.

I've tracked it down to a potential issues within protocol.py, where a deprecated EOF is being detected, which stops reading in retrieved records.

I've had to comment out this section of code, and it seems to have solved my issue. Is this a bug, or a known incompatibility?

MySQL: /usr/sbin/mysqld (mysqld 5.7.41-0ubuntu0.18.04.1)
Python 3.9.5
MySQL Connector: mysql_connector-2.2.9-py3.9.egg-info

mysql/connector/protocol.py
Line 337-340
#elif eof57 and (packet[4] == 0 and packet[0] > 9):
# EOF deprecation: make sure we catch it whether flag is set or not
#eof = self.parse_ok(packet)
#rowdata = None
↧
↧

deallocating None issue with connect/disconnect (4 replies)

May 22, 2023, 12:41 pm
$
0
0
Hi all,

I run into a "deallocating None" error, if I use connect()/disconnect():
...
'None' refcount: 4
'None' refcount: 3
'None' refcount: 2
Fatal Python error: deallocating None
Python runtime state: initialized

Current thread 0x00007f9dfb640740 (most recent call first):
File "test.py", line 19 in <module>

Any ideas? Here my environment:
python: 3.8.10
OS: ubuntu 20.04
mysql-connector-python: 8.0.33

Samplecode to reproduce, just replace mysql.connector.connect() parameter with matching once:

import mysql.connector
from mysql.connector import Error
import sys
import time

if __name__ == "__main__":
while True:
try:
print(f"'None' refcount: {sys.getrefcount(None)}")
connection = mysql.connector.connect(
host = host,
port = port,
user = user,
passwd = password,
database = dbname
)
connection.disconnect()
time.sleep(0.01)
except Error:
print("error")
break
↧

How to perform unit testing (4 replies)

June 9, 2023, 7:23 am
$
0
0
I want to know how to execute unittests.py with my own database parameters.
↧
Search

Using mysql-connector-python 8.0.33 with MySQL 8.1 (2 replies)

October 11, 2023, 5:00 am
$
0
0
Hi,

I would like to know if using mysql-connector-python 8.0.33 with MySQL 8.1 is officially supported, or if I am required to use mysql-connector-python==8.1.0.

Thanks for your help.
↧

How to set a driver-side query execution timeout (no replies)

October 31, 2023, 4:13 pm
$
0
0
Hi, is there a way to set a driver-side query execution timeout? We noticed that the MySQL JDBC driver supports setting a driver-side socket/network timeout but the MySQL Python driver does not. This is a problem for us because if the server becomes unresponsive, any queries being executed will hang indefinitely. Is there a suggested solution for this problem?
↧
↧

SSL error using mysql-connector-python 8.2.0 (no replies)

November 29, 2023, 1:44 am
$
0
0
Hello Experts,
i am using a simple python script to connect to mysql PAAS instance

import mysql.connector
from mysql.connector import errorcode
import os
try :
RPA_db = mysql.connector.connect(host="xxxx.mysql.database.azure.com", user="username",
database="rpa_log",password="YYYYYY",ssl_ca="C:\\xx\\xx\\Desktop\\Data\\xxxx.crt.pem")
except mysql.connector.Error as err:
print(str(err))
exit()
while executing the script am getting the below error

2055: Lost connection to MySQL server at xxxxxx.azure.com:3306', system error: 1 [SSL: NO_CIPHERS_AVAILABLE] no ciphers available (_ssl.c:997)

any help would be appriciated.
↧

Unexpected error when calling purge command via execute (no replies)

January 23, 2024, 10:28 am
$
0
0
Hello!

Running python3.8, mysql-connector-python 8.3.0, and mysql 8.0.35, I've run into something that I'm unsure qualifies as a bug, but at the very least I find strange behavior.

I've got some typical factory code that has 2 exactly-as-you'd-expect context-manager functions, one yields a database connection and the other uses that function to yield a cursor (for context for the next part).

-----
class DBConnectionFactory:
host: str
port: int
db: str
user: str
password: str

@contextlib.contextmanager
def connect(self) -> Iterator[mysql.MySQLConnection]:
conn = mysql.connect(
host=self.host,
port=self.port,
db=self.db,
user=self.user,
password=self.password,
)
conn.get_warnings = True
try:
yield conn
finally:
conn.close()

@contextlib.contextmanager
def cursor(self, dictionary: bool = False) -> Iterator[Cursor]:
with self.connect() as conn:
cur = conn.cursor(dictionary=dictionary)
try:
yield cur
finally:
cur.close()
-----



Attempting to run a PURGE BINARY LOGS command only works when run via the database connection:

-----
with db.connect() as conn:
conn.cmd_query(f"PURGE BINARY LOGS BEFORE '{start_time}';")
-----


Now, this is basically what the documentation dictates, but for the longest time, I was running it like this

-----
with db.cursor() as cur:
cur.execute(f"PURGE BINARY LOGS BEFORE '{start_time}';")
-----

because I did not understand that there was a difference at first. The "bug" behavior comes in with how mysql connector handles the response from the server. The purge command itself is run, I can validate it by listing the binary log files and asking mysql to SHOW BINARY LOGS. but the error comes in the stack trace.

-----
Traceback (most recent call last):
File "script.py", line 193, in flush_logs
cur.execute(f"PURGE BINARY LOGS BEFORE '{start_time}';")
File "venv/lib/python3.8/site-packages/mysql/connector/cursor.py", line 748, in execute
self._handle_result(self._connection.cmd_query(stmt))
File "/venv/lib/python3.8/site-packages/mysql/connector/cursor.py", line 614, in _handle_result
self._handle_noresultset(result)
File "/venv/lib/python3.8/site-packages/mysql/connector/cursor.py", line 585, in _handle_noresultset
self._handle_warnings()
File "/venv/lib/python3.8/site-packages/mysql/connector/cursor.py", line 1054, in _handle_warnings
warnings.warn(err, stacklevel=4)
TypeError: expected string or bytes-like object
-----

Best as I can tell, the warning handler for the cursor isn't prepared to handle the non-row response from the server. I confirmed that start_time was in fact a string, as well as the entire string I was passing to execute, so I looked into the source code and confirmed it was being thrown by the library.

Now, I'm not entirely sure what the best approach to this is. I'm unsure if it qualifies as a bug since I'm a) probably not supposed to be running the "Database level commands" like purge through execute and b) fully aware of the existence of cmd_query now

However, it seems like the library should either a) handle the error in a more pythonic manner since both approaches to actually run the command b) not let me run the command via execute or c) not throw an error at all.

I also verified that this error is only occurring after 8.0.12, as it doesn't throw an error at all with that version, but all subsequent versions throw the error. I'd be happy to file a bug request if this is considered a bug, but I figured I'd check in the forums before doing anything.
↧

MySQL Asynchronous Connectivity with MySQL Connector/Python (no replies)

March 12, 2024, 3:02 pm
↧

migrating multi-line SQL commands from pymysql (no replies)

March 28, 2024, 7:30 am
$
0
0
I've been tasked with migrating from pymsqyl to mysql.connector . . . we have a bunch of code that has multi-line SQL statements that don't run as expected in mysql.connector, viz. only the first command in the multi-line statement gets run . . . Is there a best way to migrate from pymysql to mysql.connector? I've added `client_flags=ClientFlag.MULTI_STATEMENTS` but then I seem to need to add `while connection.next_result(): pass` after every insert/update statement that has multi-line SQL before I can do a commit.

Is there a better way?
↧
↧

Python Connector gives incorrect result (no replies)

April 7, 2024, 5:43 pm
$
0
0
This python code:

```python
import mysql.connector

try:
conn = mysql.connector.connect(
host="localhost", user="fanw", password="secret", database="fanw"
)

cursor = conn.cursor()
query = """
CREATE TEMPORARY TABLE IF NOT EXISTS temp_pairs AS

SELECT
fd.nid,
fd.title
FROM
node__field_card_display cl
JOIN
node_field_data fd ON cl.entity_id = fd.nid AND cl.langcode = fd.langcode
WHERE
cl.field_card_display_target_id = 2049
AND cl.langcode = 'en'
AND cl.bundle = 'person';

SELECT
t.entity_id,
t.deleted,
t.delta,
t.langcode,
t.body_value,
GROUP_CONCAT(DISTINCT tp.nid SEPARATOR ',') AS nids,
GROUP_CONCAT(DISTINCT tp.title SEPARATOR ',') AS titles
FROM
node__body AS t
JOIN
temp_pairs AS tp ON BINARY t.body_value LIKE CONCAT('%', tp.title, '%')
AND NOT (t.body_value LIKE CONCAT('%', tp.nid, '">', tp.title, '<', '%'))
WHERE
t.body_format <> 'plain_text'
AND t.entity_id <> tp.nid
GROUP BY
t.entity_id, t.langcode;
"""
cursor.execute(query, multi=True)
result = cursor.fetchall()
print(len(result))
conn.commit()

except Exception as e:
conn.rollback()
print(f"Error: {str(e)}")

finally:
cursor.close()
conn.close()

print("DONE!")
```

Gives the result: 0

While running the same query directly in mysql gives: 51

```sh
$ cat python_query.sql
CREATE TEMPORARY TABLE IF NOT EXISTS temp_pairs AS

SELECT
fd.nid,
fd.title
FROM
node__field_card_display cl
JOIN
node_field_data fd ON cl.entity_id = fd.nid AND cl.langcode = fd.langcode
WHERE
cl.field_card_display_target_id = 2049
AND cl.langcode = 'en'
AND cl.bundle = 'person';

SELECT
t.entity_id,
t.deleted,
t.delta,
t.langcode,
t.body_value,
GROUP_CONCAT(DISTINCT tp.nid SEPARATOR ',') AS nids,
GROUP_CONCAT(DISTINCT tp.title SEPARATOR ',') AS titles
FROM
node__body AS t
JOIN
temp_pairs AS tp ON BINARY t.body_value LIKE CONCAT('%', tp.title, '%')
AND NOT (t.body_value LIKE CONCAT('%', tp.nid, '">', tp.title, '<', '%'))
WHERE
t.body_format <> 'plain_text'
AND t.entity_id <> tp.nid
GROUP BY
t.entity_id, t.langcode;

$ mysql -u fanw -p fanw < python_query.sql | wc -l
Enter password:
52 (first row are the column names)
```

```sh
$ pip freeze | grep mysql
mysql-connector-python==8.3.0
```

python version is 3.9.10

And I'm running this on a MacBook Pro M1 (12.7).

I've also run the query using DBeaver, and it gives the same result; 51.
↧
Search

ClientFlag MULTI_STATEMENTS (no replies)

April 27, 2024, 1:31 pm
$
0
0
I checked documents, but can't found sample codes of executing multi statements using mysql-connector-python, so need help to correct the following.

```
import mysql.connector
from mysql.connector import ClientFlag

# Connect with the MySQL Server
con = mysql.connector.connect(**CFG)
con.set_client_flags([ClientFlag.MULTI_STATEMENTS])
con.reconnect()

cur = con.cursor()

SQL = "update class set age=123 where name='w1';update class set age=456 where name='w2';"
cur.execute(SQL)

....
```
↧

Slow query performance (no replies)

May 26, 2024, 3:03 pm
$
0
0
Hi

I have written a Python program to strip down a csv log file that contains every connection to our streaming server.

The data is stripped from the csv log and then processed line by line and each line is stored in a table.

Every time I read a line from the csv file, I check if there isn't an existing entry like this in the database to prevent duplicates.

The problem I have is that the database table currently has over 1 900 000 records. I'm using a select query by only selecting one column to try and optimize the query and feeding it only the necessary columns to match. I only need a row count returned.

Each time I run the query, it takes about 3 to 4 seconds to return a count. Checking the CPU usage of mysql shows it maxed out to 100%

A csv log file can grow as big with 4000 new lines in one hour and will expand when we have more clients.

If one line takes 3 seconds to process, This means the current instance of the python program will complete over just 3 hours.

To add to the problem, I have a cronjob running each hour to run the python program to strip out the past hour's entries. This will slow the system even further down as there is an existing instance running of the program.

Is there any way that I can optimize the query any further or change settings in mysql or any other ways to speed up these queries?
↧

TypeError( TypeError: Python 'states' cannot be converted to a MySQL type (no replies)

June 6, 2024, 11:43 am
$
0
0
Using `mysql-connector-python` 8.4.0 with Python 3.11 and Django 4.2


I have a model `Job` and when I try to create an instance, I get the following error which seems to suggest that the connector is not able to handle the Django enumeration type as described here...

https://docs.djangoproject.com/en/3.2/ref/models/fields/#enumeration-types

What am I missing?

Rob

```Error
Traceback (most recent call last):
File "/Users/robertcollins/github/mtm-crm-patch/venv11/lib/python3.11/site-packages/mysql/connector/cursor.py", line 564, in _process_params
res = [to_mysql(value) for value in res]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/robertcollins/github/mtm-crm-patch/venv11/lib/python3.11/site-packages/mysql/connector/cursor.py", line 564, in <listcomp>
res = [to_mysql(value) for value in res]
^^^^^^^^^^^^^^^
File "/Users/robertcollins/github/mtm-crm-patch/venv11/lib/python3.11/site-packages/mysql/connector/conversion.py", line 237, in to_mysql
raise TypeError(
TypeError: Python 'states' cannot be converted to a MySQL type
```

The Django Model definition is:-
```
class Job(models.Model):
class STATES(TextChoices):
NEW = "NEW"
READY = "READY"
PROCESSING = "PROCESSING"
STOPPING = "STOPPING"
FAILED = "FAILED"
COMPLETE = "COMPLETE"

id = UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
created = models.DateTimeField(auto_now_add=True, db_index=True)
modified = models.DateTimeField(auto_now=True)
name = models.CharField(max_length=100)
state = models.CharField(
max_length=20, choices=STATES.choices, default=STATES.NEW, db_index=True
)
next_task = models.CharField(max_length=100, blank=True)
workspace = JSONField(null=True)
queue_name = models.CharField(max_length=20, default="default", db_index=True)
priority = models.SmallIntegerField(default=0, db_index=True)
run_after = models.DateTimeField(null=True, db_index=True)

class Meta:
ordering = ["-priority", "created"]

objects = JobManager()

def save(self, *args, **kwargs):
if self._state.adding:
self.next_task = get_next_task_name(self.name)
self.workspace = self.workspace or {}

try:
self.run_creation_hook()
except Exception as exception: # noqa
logger.exception(
"Failed to create new job, creation hook raised an exception"
)
return # cancel the save

return super().save(*args, **kwargs)

def update_next_task(self):
self.next_task = get_next_task_name(self.name, self.next_task) or ""

def get_failure_hook_name(self):
return get_failure_hook_name(self.name)

def get_creation_hook_name(self):
return get_creation_hook_name(self.name)

def run_creation_hook(self):
creation_hook_name = self.get_creation_hook_name()
if creation_hook_name:
logger.info("Running creation hook %s for new job", creation_hook_name)
creation_hook_function = import_string(creation_hook_name)
creation_hook_function(self)

@staticmethod
def get_queue_depths():
annotation_dicts = (
Job.objects.filter(state__in=(Job.STATES.READY, Job.STATES.NEW))
.values("queue_name")
.order_by("queue_name")
.annotate(Count("queue_name"))
)

return {
annotation_dict["queue_name"]: annotation_dict["queue_name__count"]
for annotation_dict in annotation_dicts
}
```

The Job create is...
```
context = {
"postal_code": self.applicant_postal_code,
"application_id": self.id,
}
try:
Job.objects.create(
name="locate_application_form_job",
workspace=context,
queue_name="mapit_queue",
)
except Exception as e:
logger.error(
f"Cannot create background locate_application_form_job for application"
f":{self.id}\n{e}"
)
```
↧
↧

Async connector and connection pool (1 reply)

June 10, 2024, 3:37 pm
$
0
0
Hello!
I developed an application, basted on FastAPI, currently implemented using the classic blocking MySQL Python connector, which allows multiple clients to concurrently query the database.
In order to improve performances with many active clients, I'm using a connection pool.
I would like to use the new async/await api offered by version 8.3 of the Python connector, but it seems it's lacking connection pool support.

Now, my question is the following: do I really need a connection pool?
↧

Poll: Anyone using c-extension? (no replies)

June 13, 2024, 7:51 am
$
0
0
Hey devs

Can I ask if you would reply to this thread and let me know if you are. using the c-extension and, if so, why?

Thanks!
↧

[BUG] v8.2.0 breaking with ssh tunnel (no replies)

July 25, 2024, 3:53 am
$
0
0
Hi,

Trying to run 8.2.0 of the mysql python connector with the sshtunnel package to connect to a remote server. The code is very simple and I used a new environment with bare minimum packages. It seems like 8.2.0 introduced a breaking change that makes the code hang at the mysql.connector.connect() call and times out. The same code on 8.1.0 works fine and produces the expected output. 

from sshtunnel import SSHTunnelForwarder
import mysql.connector

# Define your SSH and database connection details
SSH_HOST = ''
SSH_PORT = 22
SSH_USERNAME = ''
SSH_PASSWORD = ''

DB_HOST = 'localhost'
DB_PORT = 3306  
DB_USER = ''
DB_PASSWORD = ''

with SSHTunnelForwarder(
    (SSH_HOST, SSH_PORT),
    ssh_username=SSH_USERNAME,
    ssh_password=SSH_PASSWORD,
    remote_bind_address=(DB_HOST, DB_PORT)
) as tunnel:
    local_port = tunnel.local_bind_port

    connection = mysql.connector.connect(
        host='127.0.0.1',
        port=local_port,
        user=DB_USER,
        password=DB_PASSWORD
    )

    cursor = connection.cursor()
    cursor.execute("SHOW DATABASES")
    databases = cursor.fetchall()

    for db in databases:
        print(db[0])

    cursor.close()
    connection.close()
-----

Python version 3.12.4
Installed packages

Package Version
---------------------- -------
bcrypt 4.2.0
cffi 1.16.0
cryptography 43.0.0
mysql-connector-python 8.1.0
paramiko 3.4.0
pip 24.0
protobuf 4.21.12
pycparser 2.22
PyNaCl 1.5.0
setuptools 69.5.1
sshtunnel 0.4.0
wheel 0.43.0
↧

mysql.connect kernel has died (no replies)

October 15, 2024, 2:13 am
$
0
0
Dear all,
I am trying to connect to my database using Python commands such as:

conn= mysql.connector.connect(host='127.0.0.1',
user='root',
password='my_real_password',
database='database_name')

Unfortunately, Python kernel dies and no error message pop out.

Any advice would be greatly appreciated.

Best !
↧
Search
↧

help needed (1 reply)

October 25, 2024, 8:57 pm
$
0
0
A very noob question maybe sorry

Why am I getting a `mysql.connector.errors.InterfaceError: 2013, 'Lost connection to MySQL server during query' when executing large transactions?
↧
Viewing latest articles
Browse all 384 Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>