* added the sorting of FILES.json based on name with ASCII2 ordering
* remove useless code
* Add comprehensive tests for FILES.json sorting to ensure reproducible builds
This commit adds tests to validate the FILES.json sorting feature implementation (issue #82792).
Unit Tests (test/units/galaxy/test_collection.py):
- test_build_files_manifest_sorted_by_name: Validates ASCII sorting
- test_build_files_manifest_walk_sorted_output: Tests walk function
- test_build_collection_reproducible_build: Ensures reproducibility
- test_build_files_manifest_special_characters_sorted: Edge cases
- test_files_json_has_sorted_keys: Validates JSON key ordering
- test_build_collection_large_number_of_files_sorted: Scale testing
Integration Tests (test/integration/.../build.yml):
- Multiple builds produce identical FILES.json
- Verify FILES.json reproducible across builds
- Verify all files in ASCII sorted order
- Validate FILES.json format and structure
- Verify each file entry has required fields
Per the comment that 'this should follow the same pattern as _build_collection_tar', add `sort_keys` to this invocation of `json.dumps`
Also removes a sort because files area already sorted in `_build_files_manifest`, which is the source of `file_manifest` in `_build_collection_dir`'s only invocation (line 1669)
Co-authored-by: Thomas Wang <yuanchen.wang@optus.com.au>
Co-authored-by: Thomas Wang <yctomwang123@gmail.com>
Co-authored-by: pvskp <pvincius14@gmail.com>
Currently when the editor returns a failure exit code in `ansible-vault edit`,
the original file is still being overwritten by whatever is in the temporary
file. This is undesirable when some process already made edits to the temporary
file and then later fails for some reason, since the file might only be
partially modified and thus corrupt.
Fix this by using `check_call()` instead of `call()` to throw an Exception when
the exit code is non-zero.
Co-authored-by: Matt Clay <matt@mystile.com>
* Register projections and action plugin variable API
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Patrick Kingston <pkingsto@redhat.com>
* While creating bare minimum container images, sometimes
/var/lib/apt/lists is removed.
Recreate this directory in order to update cache successfully.
Fixes: #61176
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
Co-authored-by: Sloane Hertel <19572925+shertel@users.noreply.github.com>
* Simplify pipelining logic for Windows ci_complete
Has the Windows connection plugins override is_pipelining_enabled to
return True rather than use the special connection plugin attributes.
These attributes should be removed in the future but when is still
dependent on when we can expect Ansible 2.19 is the minimum version
supported in collections.
* Make CI green
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
---------
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
Co-authored-by: Abhijeet Kasurde <Akasurde@redhat.com>
When editable is set to true, -e flag should be passed to all packages.
This change passes -e flag before each package name. However, if a
requirements file is used, then editable flag raises errors. Therefore,
editable and requirements are now mutually exclusive.
Fixes#77755
Signed-off-by: Nirmal Patel <nirmal@nirmal.dev>
* Add support for PowerShell modules on POSIX
Adds support for running modules written in PowerShell on non-Windows
hosts. This includes references to a PowerShell or C# module_util
located in Ansible or a collection. Not all module utils will work
outside of Windows but `Ansible.Basic` will do so.
Support for PowerShell modules on non-Windows is up to the module and
collection author. This PR just enables the ability to run them through
the existing PowerShell execution wrapper.
* Fix up sanity and unit tests, try and run in separate CI group
* Fix up powershell.sh group detection
* More sanity fixes
* More sanity fixes
* Ensure shebang is part of command to run
* Try and simplify exec module logic
* Attempt to get powershell group running in CI
* Fix up test integration aliases for powershell
* Remove ansible.windows collection for integration support
* Revert the win_powershell changes now they aren't needed
* Simplify test matrix and use default container
On platforms like Alpine and BusyBox, group delete operation
with force is not applicable. Raise an error notifying the
user about the same.
Fixes: #85565
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
* split the dnf module into a cli shim and a module
* Update package_facts to use rpm cli, add package_facts and rpm_key to rhel8 targeted testing
* Switch resource embedding to EmbedManager.embed API
Co-authored-by: Matt Davis <nitzmahone@redhat.com>
* Fix using the server's validate_certs configuration when downloading collections
* Fix validate_certs for verify
There is no GalaxyAPI on the collection object for verify since it wasn't created via the resolver
Remove unit test - would need more convoluted monkeypatching
* Simplify ConcreteArtifactsManager.save_collection_source by passing 2 arguments instead of 6
* Consolidate ConcreteArtifactsManager instance attrs _galaxy_collection_cache/_galaxy_collection_origin_cache
* Add new psrp options and change service default
Add new options to the `psrp` connection plugin to specify the profile
loading behaviour and a password to use for decrypting the certificate
authentication private key.
This also changes the `negotiate_service` default from `WSMAN` to `host`
to improve compatibility with Windows targets that may not have the
`WSMAN` SPN registered like domain controllers and align with the
defaults the native PowerShell PSRemoting client uses.
* Update lib/ansible/plugins/connection/psrp.py
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
---------
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
When the search fails, as expected, the following error is printed out:
[...] Failed to read the file FILENAME due to an encoding error. current encoding: None (default determined by the Python built-in function "open")
Add locale encoding in error messages when none is given.
As well, this case is hit for decoding exceptions, not encoding ones.
Change the error message.
Add the corresponding tests, update documentation.
Link: https://docs.python.org/3/library/locale.html#locale.getpreferredencoding
Co-authored-by: Matt Clay <matt@mystile.com>
Signed-off-by: Ariel Otilibili <a.otilibili@instadeep.com>
When the seuser parameter is provided but SELinux is not enabled on the
target system, the parameter is silently ignored and the module reports
changed: true. This misleads users into thinking SELinux user mappings
were applied.
Emit a warning when seuser is set and SELinux is not enabled, so users
are aware the parameter has no effect. This uses module.selinux_enabled()
to check at runtime rather than checking the platform type, as recommended by the maintainers.
The warning alone is not enough — useradd still receives the -Z flag
and fails with "useradd: -Z requires SELinux enabled kernel". Skip
adding -Z to the command when SELinux is not enabled.
Fixes#85542
---------
Co-authored-by: olegnazarov23 <olegnazarov23@users.noreply.github.com>
Fixes the action plugin mkdtemp and expand_user calls to fallback to the
original non-pipelined variants when a connection plugin does not
support pipelining. For example a 3rd party Windows connection plugin
may not support pipelining and thus cannot use the user _mkdtemp2 and
_expand_user2 variants exposed by the `powershell` shell plugin as they
require data to be sent over stdin.
* Raise warning on inaccessible path (#57573)
* Raise error on permission denied (#57573)
* Use `os.lstat` for all state operations in `file`
* Update warning messages
* Update changelog
* Make tests use remote_tmp_dir for easier cleanup
* Rename and update changelog fragment
* Tweak changelog fragment
---------
Co-authored-by: Jakub Pieńkowski <8525083+Jakski@users.noreply.github.com>
* ansible-galaxy - only install/download collections with compatible requires_ansible metadata
inject requires_ansible as a dependency of the collection
add a hack to preserve the collection origin in the error message in get_dependencies and find_matches
* yield requires_ansible requirement before dependencies
* Fix error handling and broken test
Fix getting requires_ansible dependency after the metadata is available
Allow 3rd party servers to provide this metadata lazily for individual versions
Rename the candidate attr name for the error handling hack
* fix diff
* clarify code comments
refactor a bit
* Don't duplicate error message. Before:
[ERROR]: Failed to resolve the requested dependencies map. Could not satisfy the following requirements:
* ns.col2:1.0.0 (dependency of ns.col3:1.0.0) requires ansible-core <2.19.1
Hint: To disregard whether the collection supports the current version of ansible-core, configure COLLECTIONS_ON_ANSIBLE_VERSION_MISMATCH as "ignore".
Hint: Pre-releases hosted on Galaxy or Automation Hub are not installed by default unless a specific version is requested. To enable pre-releases globally, use --pre: [RequirementInformation(requirement=<ansible-core:<2.19.1 of type 'requires_ansible' from Galaxy>, parent=<ns.col2:1.0.0 of type 'galaxy' from cmd_arg>)]
Failed to resolve the requested dependencies map. Could not satisfy the following requirements:
* ns.col2:1.0.0 (dependency of ns.col3:1.0.0) requires ansible-core <2.19.1
Hint: To disregard whether the collection supports the current version of ansible-core, configure COLLECTIONS_ON_ANSIBLE_VERSION_MISMATCH as "ignore".
Hint: Pre-releases hosted on Galaxy or Automation Hub are not installed by default unless a specific version is requested. To enable pre-releases globally, use --pre.
<<< caused by >>>
[RequirementInformation(requirement=<ansible-core:<2.19.1 of type 'requires_ansible' from Galaxy>, parent=<ns.col2:1.0.0 of type 'galaxy' from cmd_arg>)]
After:
[ERROR]: Failed to resolve the requested dependencies map. Could not satisfy the following requirements:
* ns.col2:1.0.0 (dependency of ns.col3:1.0.0) requires ansible-core <2.19.1
Hint: To disregard whether the collection supports the current version of ansible-core, configure COLLECTIONS_ON_ANSIBLE_VERSION_MISMATCH as "ignore".
Hint: Pre-releases hosted on Galaxy or Automation Hub are not installed by default unless a specific version is requested. To enable pre-releases globally, use --pre.
* appease CI
fix type hints
* Update changelogs/fragments/install-ansible-core-compatible-collections.yml
Co-authored-by: sivel / Matt Martz <matt@sivel.net>
* Hardcode unsupported requires_ansible version to make maintaining the tests easier
Add a test for failed backtracking
Co-authored-by: sivel / Matt Martz <matt@sivel.net>
---------
Co-authored-by: sivel / Matt Martz <matt@sivel.net>
* Fix vaulted value rendering
- Fixes AnsibleDumper works on vaulted strings
- Adds config setting for opting in to future forced-strictness
* Add tests for new to_yaml functionality
* Add return on all paths
* Clarify configured value name
* Fix sanity (remove unused import)
* Catch more cases of encrypted strings
* WIP fix failing unit tests
* Update unit test to work with new to_yaml features
* Remove unit test
Running coverage with/without shows no new coverage.
* Write some basic error text
* Fix tests for error configuration
* Change names
- Change `vault_behavior` to just `vault`
- Change `preserve` to `keep_encrypt`
* Fix small issues in tests
* Fix fmt with black
* Fix unit tests
- Fix imports
- Fix the name of the param
* Update assorted docs - code review
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* Remove deprecation
* Simplify vault parameter and handling
- Get rid of `default` enum
- Get rid of config option and associated test file
- Change `vault` to `vault_behavior`
* Document filter parameter
* Correctly Document filter parameter
* Add version_added and changelog fragment
* Tweak changelog
* Remove dead code, add docstring
* Add bugfixes section to changelog
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
---------
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
* Replace gpg in rpm_key with librpm
* Manually compute primary key ID and fingerprint for older librpm
* Code cleanup
* Add some v6 tests
* Add rhel 10.1 to CI matrix and changelog
* Remove RHEL 10.0
* consolidate common code and rename unused vars
* remove unnecessary exception raise
* Fix return type
* review comments: change import and use of Optional
* ci_complete ci_coverage
* address review comments
* When checking for existing keys, account for short from key ID and revert test change that hid this
* Support RPM version 6+. Allow 'key' to be fingerprint.
* Replace v6 test key with non-PQC algo version and enable Fedora v6 tests
* modify changelog, mod doc, and del 10.1 test req
* Code refactor and new tests
Refactor code to not need to use librpm API to get list of installed
keys. Also add new tests to verify deleting by fingerprint.
* Refactor drop_key() by rpm version, fully type hint code
* use hexdigest()
sivel / Matt Martz