Description

The invention relates to an apparatus and a method for the automated bending of workpieces.

Apparatuses for the automated bending of workpieces using a robotic device for handling the workpieces are known from the prior art. A workpiece to be handled is automatically removed from a storage by the robotic device and fed to a bending machine, which causes the workpiece to be deformed by a bending process. In order for the robotic device to be able to correctly remove the corresponding workpiece from the storage, image data from an image capture device are used, from which the location of the workpiece to be handled in the storage is identified. The robotic device is then suitably controlled by means of the identified location.

Document WO 2020/250761 Al discloses an apparatus for the automated bending of workpieces, with images of a workpiece stack in a storage being photographed by a monocular camera and evaluated by means of pattern recognition. The location of the top workpiece on the workpiece stack is detected using a plurality of models.

Document WO 2021/079802 Al describes a method for the automated bending of workpieces in which images that are photographed by a camera above the position of a workpiece are used to determine the location of the workpiece and, on this basis, a robotic device for transporting the workpiece is suitably controlled.

Conventional apparatuses for the automated bending of workpieces have the disadvantage that the image capture device which is used for detecting workpieces to be handled in a storage is arranged at a fixed position, so the field of view of the image capture device is static and only a limited spatial region of workpieces to be handled can be captured.

Document US 2020/023521 Al discloses a method of controlling a robot system. In a calibration process, a hand camera attached to a robot hand captures an image of reference markers. Based on the captured image, calibration data between a robot coordinate system and a marker coordinate system are determined. For the normal operation of the robot, the hand camera is detached from the robot hand so that the robot hand can handle workpieces. During the normal operation, a fixed camera captures an image of a corresponding workpiece and the reference markers, and an image processing device calculates the relative position between the workpiece and the robot based on the captured image and the calibration data.

Document DE 10 2017 123 877 Al discloses a robotic system for a shape forming machine, the robotic system comprising a manipulator for manipulating a workpiece. A camera coupled with the manipulator captures the relative position between at least one marker and the manipulator. A calculation unit connected to the camera calculates from the relative position at least one correction value for controlling the movement of the manipulator. Document US 8 798 794 B2 discloses a method for precisely positioning at least one object in an end position in space by means of an industrial robot. The method uses a 3D image recording device comprising an angle measuring unit. The position of the object is determined based on the position of the 3D image recording device, the angular orientation of the 3D image recording device which is detected by the angle measuring unit, a 3D image recorded by the 3D image recording device and the knowledge of features on the object.

The object of the invention is to create an apparatus and a method for the automated bending of workpieces, with which workpieces to be handled can be captured in a larger spatial region.

This object is achieved by the apparatus according to claim 1 and the method according to claim 15. Further developments of the invention are defined in the dependent claims.

The apparatus according to the invention is used for the automated bending of workpieces, preferably metallic workpieces such as metal sheets. The apparatus comprises a storage for the corresponding workpieces, a robotic device for handling the workpieces and a bending machine for deforming at least some of the workpieces handled with the robotic device via a bending process. The bending machine is preferably a press brake.

The apparatus according to the invention also includes an image capture device for capturing image data about workpieces in the storage which are to be handled and a control device for controlling the robotic device using said image data.

In the apparatus according to the invention, the image capture device can be moved into different positions via an actuator system in order to capture image data from different sections of the storage which are associated with the respective positions. The actuator system can preferably be controlled via the above control device. Furthermore, a plurality of markings are provided on the storage, the positions of which relative to a reference coordinate system are stored in the control device, i.e. the positions of the markings are already known. Preferably, the positions of the markings are three-dimensional spatial positions. If necessary, however, the positions can also be two-dimensional positions, for example if the height positions of workpieces to be handled are obtained from an information source other than the image data. The reference coordinate system is a coordinate system that is stationary in relation to the apparatus according to the invention and does not follow relative movements of components within the apparatus, such as the movements of the robotic device.

According to the invention, at least two of the plurality of markings are contained in a respective position of the image capture device in the associated storage section, and the control device is configured to carry out an evaluation of the image data of the associated storage section, in which the location of the image capture device is automatically determined using the stored positions of the markings in the associated storage section and/or the location of a workpiece to be handled relative to the reference coordinate system is automatically determined. The location of the image capture device or of the workpiece to be handled describes the position and orientation of the image capture device or of the workpiece to be handled, the position preferably being a three-dimensional spatial position and the orientation preferably being a three-dimensional spatial direction.

The markings used in the method according to the invention can be passive and/or active markings. No information about their position can be read from passive markings. For this reason, before the method according to the invention is carried out, the position of a respective passive marking relative to the reference coordinate system has been determined using a suitable measurement and stored in the control device. In contrast to this, information about their positions is stored in active markings. Before the method according to the invention is carried out, this information is read from the active markings in order to then be assigned to the reference coordinate system in the control device and stored there.

The apparatus according to the invention enables a larger field of view of a corresponding workpiece storage by positioning the image capture device in different positions. At the same time, an external calibration of the image capture device in its current position is made possible, with suitable markings on the storage being used for this purpose. The external calibration ensures that for each position of the image capture device its location in space is known. Therefore, a larger spatial region of workpieces can be captured by moving the image capture device.

In a preferred embodiment of the apparatus according to the invention, the location of a workpiece to be handled in the associated storage section relative to the reference coordinate system is determined from the location of the image capture device. This location can then be used to handle corresponding workpieces. The location of the workpiece to be handled describes its position and orientation, the position preferably being a three-dimensional spatial position and the orientation preferably being a three-dimensional spatial direction.

In a particularly preferred embodiment, the image capture device is a 3D image capture device, i.e. an image capture device with which three-dimensional image data are captured. The 3D image capture device is preferably a 3D camera device. If necessary, a 2D image capture device can also be used in the apparatus according to the invention. This is the case, for example, when the height positions of workpieces are obtained from an information source other than the image data.

In a further preferred embodiment of the apparatus according to the invention, separate storage regions for workpieces are assigned to the storage sections and a respective storage section completely covers the assigned storage region vertically in plan view. Separate storage regions are understood to be spatially disjoint storage regions. In contrast to this, the storage sections defined above with the markings contained therein do not have to be spatially disjoint. That is, a storage section can be larger than a storage region and the same markings can belong to a plurality of storage sections.

In a preferred variant of the embodiment just described, a storage region is a carrier for storing workpieces, with the carrier preferably being a pallet. Furthermore, the workpieces can preferably be stacked in the corresponding storage region.

In a further preferred embodiment of the apparatus according to the invention, the storage sections with the markings contained therein are arranged next to one another in a predetermined direction.

In a further variant, the image capture device can be moved linearly by means of the actuator system, as a result of which the position of the image capture device can be changed in a simple manner. The linear movement preferably takes place in the predetermined direction defined above, in which the storage sections are arranged next to one another.

In a further, particularly preferred embodiment of the apparatus according to the invention, all markings of the plurality of markings are arranged on the floor of the storage and/or up to 500 mm above the floor, wherein the image capture device can be moved above the storage. As a result, the capture of the markings by means of the image capture device can be ensured in a particularly reliable manner. The markings do not all have to be on the same level. Rather, the markings can be positioned in several different planes. This increases accuracy in the vertical direction.

The number of storage sections provided in the apparatus according to the invention can be chosen differently. The higher this number, the larger the spatial region in which stored workpieces can be captured. In a preferred variant, at least three storage sections, preferably 3 to 10 storage sections and particularly preferably four storage sections, are provided. 1

In a further preferred variant, the at least two markings in a respective storage section represent at least some of the storage sections and preferably a subset of the plurality of markings in each storage section. The markings thus differ at least in part between the storage sections.

In a further preferred variant, the at least two markings in a respective storage section are at least part of the storage sections and preferably in each storage section at least three markings, preferably four markings. The use of more than two markings improves the accuracy of position determination of the image capture device.

In a further preferred embodiment of the apparatus according to the invention, one or more markings and preferably two markings in a respective storage section belong to at least some of the storage sections and preferably in each storage section also to a storage section other than the particular storage section. At least some of the markings are thus used simultaneously for a plurality of storage sections so that the number of markings can be reduced.

In a further preferred embodiment, a respective storage section of at least some of the storage sections and preferably each storage section has a polygonal and in particular rectangular outline in vertical plan view. With such outlines, the evaluation of the image data for determining the location of the image capture device is simplified and, if necessary, enables the direct determination of the location of the workpieces to be handled. A marking is preferably provided in one or more corners and in particular in each corner of the polygonal outline.

In a further preferred embodiment, the markings are designed to be optically different in order to be distinguishable by the control device. They preferably contain an optical code. As a result, the evaluation of the image data can be simplified and the accuracy of position determination of the image capture device can be improved. The optically different markings can, for example, have different colours. If the markings contain an optical code, it is preferably a binary code that can be processed very easily in the control device of the apparatus according to the invention.

In a further preferred variant, a respective marking comprises at least some of the plurality of markings and in particular each marking comprises at least two circle segments with a common segment centre. Using such markings, their position can be extracted very precisely from the image data and compared with the positions stored in the control device. As a result, the accuracy of position determination of the image capture device can be further improved.

The invention also relates to a method for the automated bending of workpieces with the apparatus according to the invention or one or more preferred variants of this apparatus. In other words, an apparatus according to the invention is used in the method, which comprises a storage for workpieces, a robotic device for handling the workpieces and a bending machine for deforming at least some of the workpieces handled with the robotic device via a bending process, the apparatus also comprising an image capture device for capturing image data about workpieces in the storage which are to be handled and a control device for controlling the robotic device using the image data.

As part of the method according to the invention, the image capture device is moved into different positions via an actuator system, which is preferably controlled via the above control device, in order to capture image data from different sections of the storage which are associated with the respective positions, with a plurality of markings being provided on the storage, the positions of which relative to a reference coordinate system are stored in the control device, wherein at least two markings of the plurality of markings are contained in a respective position of the image capture device in the associated storage section. In addition, the control device in the method according to the invention carries out an evaluation of the image data of the associated storage section, in which the location of the image capture device is determined relative to the reference coordinate system using the stored positions of the markings in the associated storage section.

All of the additional statements made above in connection with the explanation of the apparatus according to the invention can also be applied to the method according to the invention. In particular, the location of the image capture device describes its position and orientation, the position preferably being a three-dimensional spatial position and the orientation preferably being a three-dimensional spatial direction. Furthermore, the location of a workpiece to be handled in the associated storage section relative to the reference coordinate system is preferably determined from the location of the image capture device. This location can then be used to handle corresponding workpieces. The location of the workpiece to be handled describes its position and orientation, the position preferably being a three-dimensional spatial position and the orientation preferably being a three-dimensional spatial direction.

Exemplary embodiments of the invention are described in detail below with reference to the accompanying figures.

In the drawings:

Fig. 1 shows a schematic representation of the structure of a variant of an apparatus according to the invention;

Fig. 2 shows a schematic side view of a storage for workpieces with an image capture device arranged above it according to a preferred embodiment of the apparatus according to the invention;

Fig. 3 shows a plan view of the storage from Figure 2; and

Fig. 4 shows a flow chart illustrating the steps performed by an embodiment of the apparatus according to the invention. Figure 1 shows a schematic view of an embodiment of an apparatus according to the invention for the automated bending of workpieces. The apparatus is denoted by reference sign 1 and comprises, in a manner known per se, a storage 2 which, in the embodiment shown, contains two storage sections 201 and 202. Corresponding pallets are provided in the storage sections, on which pallets the workpieces to be handled, in the form of metal sheets, can be stacked. Above the storage 2 is a movable image capture device 7 in the form of a 3D camera, which can be moved along a linear suspension by means of an actuator system 8. In other words, the suspension contains a corresponding actuator system 8, which enables the movement of the image capture device 7 along the suspension.

A robotic device 3, which can be moved along a guide 4, is provided for the automatic handling of corresponding workpieces in the storage 2. The robotic device 3, shown here only schematically, can automatically remove workpieces from the storage via an articulated assembly (not shown), for example by means of a gripper or via suction cups. A removed workpiece is then transported by the robotic device 3 to a bending machine 5 known per se, which in turn is indicated only schematically in Figure 1. The robotic device 3 feeds the workpiece to be handled to the bending machine 5, which causes the deformation of the workpiece by exerting force on it via bending beams. The workpiece can then be placed in a corresponding storage for deformed workpieces, wherein this storage can also be contained in the storage 2.

In order to ensure the automatic handling of the workpieces by the robotic device 3, a control device 6 is provided, which evaluates image data ID captured with the image capture device 7. It should be noted here that a plurality of markings with already known three-dimensional positions are provided within the storage 2, the markings with their already known positions being taken into account when the image data ID are processed by the control device 6. For reasons of clarity, the markings cannot be seen in the schematic representation of Figure 1. However, the markings result from the embodiment of an apparatus according to the invention described further below (see in particular Figure 3).

By evaluating the image data ID with the markings contained therein, the current three-dimensional location CL of the image capture device 7 in relation to a stationary reference coordinate system RC can be determined by comparing the markings contained in the image data with their already known positions. Methods known per se can be used for this. In this way, external calibration can be carried out for any positions of the image capture device 7. With the determined position CL of the image capture device 7, the three-dimensional position WL of a corresponding workpiece to be gripped by the robotic device 3 in the stationary reference coordinate system RC can then be determined again using known methods by further evaluation of the captured image data ID. Using this position information, the robotic device 3 can be suitably controlled with the control device 6 in order to remove the corresponding workpiece from the storage 2 and feed it to the bending machine 5 in order to carry out its deformation.

Figure 2 shows a side view of a storage 2 in a variant of an apparatus according to the invention. In contrast to the apparatus in Figure 1, the storage 2 contains a total of four storage sections 201, 202, 203 and 204. According to the Cartesian coordinate system of x, y and z axes shown in Figure 2, which is also reproduced in Figure 3 described below, the storage sections are arranged next to one another in the y direction of the coordinate system. Each storage section includes a corresponding storage region 9 in the form of a pallet on which corresponding workpieces 10, in the form of metal sheets, can be stacked. As an example, two stacks of workpieces 10 are indicated for the third storage region from the left.

In the vertical direction (i.e. in the z direction of the coordinate system shown) above the storage 2 is the linear actuator system 8 already mentioned above, with which the image capture device 7 can be moved linearly in the y direction. The direction of movement of the image capture device is indicated by the arrow P. In Figure 2, the image capture device 7 is at position CPI above the left pallet, whereas at the position CP2 it is above the third palette from the left. The detection region DE of the image capture device 7 is indicated as an example for the position CP2. The planar coverage of the detection region in a vertical plan view correlates with the corresponding storage sections 201 to 204, as can be seen from Figure 3 described further below.

According to Figure 2, workpieces 10 can be captured for each of the storage sections 201 to 204 by appropriately positioning the image capture device 7 above the respective storage sections. It is thus possible to handle workpieces in a region that is extensive in the y direction using corresponding image data of the storage sections by means of the robotic device 3.

Figure 3 again shows the four storage sections 201, 202, 203 and 204 in a plan view from above. As can be seen, the storage sections have a rectangular outline. The storage section 203 corresponding to the plan view of the detection region DE from Figure 2 is indicated with dashed lines. The other storage sections are represented by dotted lines. Each storage section correlates with a position of the image capture device 7 centrally above the corresponding storage section. As can be seen clearly from Figure 3, each of the storage sections 201 to 204 contains a storage region 9 in the form of a rectangular pallet, with each pallet being uniquely assigned to a storage section. In addition, markings Ml, M2, ..., M10 with already known three- dimensional spatial positions PO1, PO2, ..., PO10 are provided in the corners of the storage sections 201 to 204. The markings and their positions are stored in the control device 6 as digital data.

Each of the markings Ml to M10 is formed by two opposing black circle segments with a common centre. This optical structure enables a very precise identification of the markings within the image data ID captured by the image capture device 7. Each of the storage sections 201 to 204 includes four markings in the corners of its rectangular outline. The storage sections overlap with one another, such that two markings for each storage section also belong to an adjacent storage section. Specifically, the storage section 201 includes markings Ml, M2, M3, and M4. The storage section 202 includes markings M3, M4, M5 and M6, with markings M3 and M4 also belonging to storage section 201. Storage section 203 includes markings M5, M6, M7 and M8, with markings M5 and M6 also belonging to storage section 202. Storage section 204 includes markings M7, M8, M9 and M10, with markings M7 and M8 also belonging to storage section 203.

In order to handle a workpiece in the storage section 203, for example, the image capture device 7 moves into the position CP2 shown in Figure 2 by means of the linear actuator system 8. Image data ID are then captured via the image capture device 7. Said image data include the markings M5, M6, M7, and M8. By comparing the already known positions of the markings with their positions in the image data, the exact three-dimensional location of the image capture device 7 in its position CP2 can be determined in a manner known per se. In other words, an external calibration of the image capture device 7 can be carried out instantaneously. By means of this external calibration, i.e. using the position of the image capture device 7, the three- dimensional location of a workpiece to be handled on the upper side of a corresponding stack can then be determined with the control device 6 by an evaluation of the image data ID which is known per se, and the robotic device for handling the workpiece can be suitably controlled.

In the embodiment described here, the number of markings can be kept low, since a plurality of markings are used simultaneously for adjacent storage sections. Furthermore, the shape of the markings ensures their highly precise identification in the image data ID and thereby improves the location determination of the image capture device 7.

Figure 4 again illustrates the essential steps that are carried out by the apparatus described above in connection with the automated bending of a workpiece.

According to step SI, the image capture device 7 is first moved into the position of that storage section in which a workpiece is to be gripped with the robotic device 3 (e.g. position CP2 in Figure 2). In step S2, three-dimensional image information ID of the storage section is then captured with the image capture device 7. In addition to the workpiece to be gripped, this image information also contains the corresponding four markings. If the image capture device 7 is in the position CP2, these are the markings M5, M6, M7 and M8. The three-dimensional location CL of the image capture device 7 relative to the above-mentioned stationary reference coordinate system RC is then determined in step S3 using a method known per se. This step thus achieves an external calibration of the image capture device in the associated position.

Finally, in step S4, the three-dimensional location WL of the workpiece to be gripped relative to the reference coordinate system RC is determined using the three- dimensional location CL of the image capture device 7, and the robotic device 3 is controlled on the basis of the location WL. In other words, the robotic device moves into the storage 2 by means of its articulated assembly and removes the workpiece to be gripped from the corresponding stack, for example via suction cups. The robotic device 3 then feeds the removed workpiece to the bending machine 5 in order to carry out the bending process.

The embodiments of the invention described above have a number of advantages. In particular, a movable image capture device makes it possible to significantly broaden the field of view of stored workpieces so that, within the context of the automated handling of workpieces, a storage for the workpieces that is significantly larger in area can be used without having to use a plurality of image capture devices. In this way, it is also ensured that after the image capture device has been moved, it is correctly calibrated, which is achieved by means of external calibration via suitable markings.

The markings can be designed differently depending on the embodiment. It is crucial that they can be identified in the corresponding image data from the image capture device. The markings do not necessarily have to be distinguishable from one another. Nevertheless, the markings can also be designed in such a way that they can be distinguished from one another optically, which can be achieved, for example, by means of optical coding. In a preferred embodiment, markings are used simultaneously by different storage sections so that their number can be reduced.

List of reference signs

1 Apparatus for the automated bending of workpieces

2 Storage

201, 202, 203, 204 Storage sections

3 Robotic device

4 Guide

5 Bending machine

6 Control device

7 Image capture device (camera)

8 Actuator system

9 Storage regions (carriers)

10 Workpieces (metal sheets)

RC Reference coordinate system

ID Image data

CL Location of the image capture device

WL Location of the workpiece to be handled/gripped

CPI, CP2 Positions (camera positions)

DE Detection region of the image capture device

P Predetermined direction (direction of movement of the image capture device)

Ml, M2, . . . , M10 Markings (passive or active)

PO1, PO2, . . ., PO10 Positions of markings

SI, S2, S3, S4 Steps