AWS SDK for pandas 3.9.1

Bug fixes 🐛

• bucketing error with newer version of Modin (0.31.0) by @LeonLuttenberger in #2896
• athena.read_sql_query failing for time columns by @LeonLuttenberger in #2895
• add an argument to control handling nulls in merge criteria by @brendan-cook-87 in #2892
• address Ray deprecation warnings by @LeonLuttenberger in #2929
• Add deprecation notice to s3.select_query by @LeonLuttenberger in #2928

Documentation 📚

• Add steps to install "using pip" to the "AWS Glue Python Shell Jobs" part of the install docs by @mc51 in #2911

Security Dependency Updates 🛡️

• bump aiohttp from 3.10.1 to 3.10.2 by @dependabot in #2930

New Contributors

• @brendan-cook-87 made their first contribution in #2892

Full Changelog: 3.9.0...3.9.1

AWS SDK for pandas 3.9.0

Enhancements 🎉

• Support ORC and CSV in redshift.copy_from_files function by @LeonLuttenberger in #2849
• Support different merge conditions in athena.to_iceberg function by @aldder in #2861
• Manage NULL values in athena.to_iceberg merge statement by @aldder in #2872
• Upgrade Ray to 2.30 by @jaidisido in #2870

Bug fixes 🐛

• Fix tz attribute check, it was checking dtype instead of dt by @sanrodari in #2855
• PyArrow backend trimming timestamp to date by @LeonLuttenberger in #2875
• Replace deprecated ray parallelism arg with override_num_blocks by @kukushking in #2876

Documentation 📚

• Adding NotebookVersion Parameter as specified in official AWS Docs by @DaxterXS in #2867
• Fix default value in doc string for athena_query_wait_polling_delay parameters by @niklasr22 in #2873

Security Dependency Updates 🛡️

• Bump urllib3 from 1.26.18 to 1.26.19 by @dependabot in #2859
• Bump certifi from 2024.6.2 to 2024.7.4 by @dependabot in #2886

New Contributors

• @aldder made their first contribution in #2861
• @DaxterXS made their first contribution in #2867
• @sanrodari made their first contribution in #2855
• @niklasr22 made their first contribution in #2873

Full Changelog: 3.8.0...3.9.0

AWS SDK for pandas 3.8.0

Enhancements 🎉

• support client-side parameter resolution in athena.create_ctas_table by @LeonLuttenberger in #2797
• add commit_transaction to postgres.to_sql by @jaidisido in #2795
• add columns parameters support by @jaidisido in #2814
• add overwrite_method to postgresql.to_sql by @LeonLuttenberger in #2820
• add upsert mode to sqlserver.py and corresponding tests by @AntonMantulo in #2835

Bug fixes 🐛

• correct athena timezone TypeError message by @jaidisido in #2796
• update license reference by @emmanuel-ferdman in #2798
• add missing index in athena.to_iceberg to_parquet by @jaidisido in #2799
• support json type in athena2pandas by @DavidKatz-il in #2806
• adapt parquet datasource to ray 2.20 by @jaidisido in #2809
• add primary keys when upserting in Postgres by @AntonMantulo in #2819
• add s3_output parameter to athena.delete_from_iceberg_table by @LeonLuttenberger in #2829
• fix s3.read_parquet_metadata not throwing proper error for empty path by @LeonLuttenberger in #2843

Documentation 📚

• fix enumerations in doc pages by @LeonLuttenberger in #2821
• fix readthedocs errors by pinning sphinx version by @LeonLuttenberger in #2822

Security Dependency Updates 🛡️

• bump werkzeug from 3.0.1 to 3.0.3 by @dependabot in #2807
• bump jinja2 from 3.1.3 to 3.1.4 by @dependabot in #2808
• bump requests from 2.31.0 to 2.32.0 by @dependabot in #2827
• bump pymysql from 1.1.0 to 1.1.1 by @dependabot in #2830

New Contributors 👏🏼

• @emmanuel-ferdman made their first contribution in #2798
• @DavidKatz-il made their first contribution in #2806
• @AntonMantulo made their first contribution in #2819

Full Changelog: 3.7.3...3.8.0

AWS SDK for pandas 3.7.3

Bug fixes 🐛

• Iceberg schema evolution fails for map, array and struct types by @LeonLuttenberger in #2755
• trickle down s3_output in athena.to_iceberg by @jaidisido in #2767
• respect order of columns in to_iceberg by @jaidisido in #2768
• add PyArrow fixed_size_binary dtype support by @jaidisido in #2775
• Opensearch serverless vector search collections - remove default _id by @kukushking in #2784
• missing keys in list_to_arrow_table by @kukushking in #2778
• prevent athena.to_iceberg overwrite to delete table in order to preserve Iceberg transactions history by @erwan-simon in #2776

Documentation 📚

• simplify README, remove AWS Glue for Ray references by @jaidisido in #2750
• fix YAML formatting in Ray Remote tutorial by @LeonLuttenberger in #2770

Security Dependency Updates 🛡️

• bump idna from 3.6 to 3.7 by @dependabot in #2772
• bump aiohttp from 3.9.3 to 3.9.4 by @dependabot in #2777

New Contributors 👋

• @erwan-simon made their first contribution in #2776

Full Changelog: 3.7.2...3.7.3

AWS SDK for pandas 3.7.2

Features/Enhancements 🚀

• Add support for DeltaLake's DynamoDB lock mechanism by @LeonLuttenberger in #2705

Bug fixes 🐛

• wr.athena.to_iceberg - Insert query has mismatched column types #2678 by @GalvFionic in #2715
• allow s3_output in athena.to_iceberg by @jaidisido in #2727
• replace deprecated np.split_array by @jaidisido in #2735
• Athena to_iceberg fails with non-lowercase column names by @LeonLuttenberger in #2736
• Support Ray 2.10 by @kukushking in #2741

New Contributors

• @GalvFionic made their first contribution in #2715

Full Changelog: 3.7.1...3.7.2

AWS SDK for pandas 3.7.1

Bug fixes 🐛

• fix breaking change in _create_table by @jaidisido in #2711
• pin pyarrow to version 8 and above by @jaidisido in #2709

Documentation 📚

• fix redshift.to_sql doc indentation error by @LeonLuttenberger in #2706

Full Changelog: 3.7.0...3.7.1

AWS SDK for pandas 3.7.0

Breaking changes 💥

Lake Formation Governed tables are being phased out and we are dropping support (#2692).

Features/Enhancements 🚀

• support parquet client encryption (#2642) by @Marwen94 in #2674

Bug fixes 🐛

• Index columns removed on s3.to_parquet by @robert-schmidtke in #2655
• Missing timezone metadata by @kukushking in #2682
• remove enforced openpyxl engine constraint by @jaidisido in #2696
• Iceberg partitioning not working with partition transform functions by @LeonLuttenberger in #2694
• remove awswrangler README from site-packages folder by @AlJohri in #2698
• indent categories in pyarrow_additional_kwargs correctly by @jaidisido in #2701

New Contributors

• @Marwen94 made their first contribution in #2674
• @AlJohri made their first contribution in #2698

Full Changelog: 3.6.0...3.7.0

AWS SDK for pandas 3.6.0

Features/Enhancements 🚀

• Enable Iceberg row deletion & add mode parameter to to_iceberg by @LeonLuttenberger in #2632
• Add support for pyarrow type large_string by @joakibo in #2663
• Add max_results to athena.list_query_executions by @LeonLuttenberger in #2665

Bug fixes 🐛

• Pyarrow 15 imports & remove unused code by @kukushking in #2649

New Contributors

• @joakibo made their first contribution in #2663

Full Changelog: 3.5.2...3.6.0

AWS SDK for pandas 3.5.2

Bug fixes 🐛

• DynamoDB key & filter expressions attribute overwrite by @kukushking in #2615
• Allow PostgreSQL reserved keywords as column names by @LeonLuttenberger in #2619
• Add to_iceberg support for filling missing columns in the DataFrame with None by @LeonLuttenberger in #2616
• Forward ignore_nulls for container types by @raaidarshad in #2636

Documentation 📚

• Add s3_additional_kwargs to docstrings by @malachi-constant in #2627
• Fix outdated hyperlinks in documentation by @LeonLuttenberger in #2634

Other 🤖

• Enable dependabot to upgrade GitHub actions by @LeonLuttenberger in #2618
• Update badges in README by @LeonLuttenberger in #2628
• Add vulnerability label to dependabot PRs with alert state by @jaidisido in #2629

New Contributors

• @raaidarshad made their first contribution in #2636

Full Changelog: 3.5.1...3.5.2

AWS SDK for pandas 3.5.1

Bug fixes 🐛

• Deserialization error when reading from DynamoDB using KeyConditionExpression by @LeonLuttenberger in #2607
• Reading of chunked parquet when columns parameter is specified by @rchromik in #2599

Documentation 📚

• Add show_create_table to Athena API page by @MikeSchriefer in #2610

Other 🤖

• chore: Replace bump2version with bump-my-version by @LeonLuttenberger in #2608
• chore(deps-dev): bump jinja2 from 3.1.2 to 3.1.3 by @dependabot in #2609
• chore(deps): bump grpcio from 1.51.3 to 1.53.0 by @dependabot in #2612

New Contributors

• @MikeSchriefer made their first contribution in #2610
• @rchromik made their first contribution in #2599

Full Changelog: 3.5.0...3.5.1