DETAILED ACTION
Notice of Pre-AIA or AIA Status
Claims 1-20 are pending in this application. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Foreign Priority
This application repeats a substantial portion of prior Application No. 17/643,185, filed December 8, 2021,with foreign priority to CN 201910568901.3 filed on June 27, 2019 and adds disclosure not presented in the prior application. Because this application names the inventor or at least one joint inventor named in the prior application, it may constitute a continuation-in-part of the prior application. Should applicant desire to claim the benefit of the filing date of the prior application, attention is directed to 35 U.S.C. 120, 37 CFR 1.78, and MPEP § 211 et seq. The presentation of a benefit claim may result in an additional fee under 37 CFR 1.17(w)(1) or (2) being required, if the earliest filing date for which benefit is claimed under 35 U.S.C. 120, 121, 365(c), or 386(c) and 1.78(d) in the application is more than six years before the actual filing date of the application. Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d) to CN 202411402380.1 filed on October 9, 2024. In reviewing the pending claims, the limitations are directed towards features that are fully disclosed and supported in the international application CN 202411402380.1 filed on October 9, 2024 and the application has been examined in accordance with a priority date of October 9, 2024, and the certified copy has been filed in the instant application.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 13 and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Folliot et al. (US PGPub US 20190287009 A1, hereby referred to as “Folliot”).
Consider Claims 1, 13 and 20.
Folliot teaches:
1. A system, comprising: at least one storage device storing executable instructions; / 13. A method / 20. A non-transitory computer readable medium, comprising at least one set of instructions, (Folliot: abstract, An embodiment provides a method for controlling scene detection by a device from among a set of possible reference scenes. The method includes detection of scenes from among the set of possible reference scenes at successive instants of detection using at least one classification algorithm. Each new current detected scene is assigned an initial probability of confidence. The initial probability of confidence is updated depending on a first probability of transition from a previously detected scene to the new current detected scene. A filtering processing operation is performed on these current detected scenes on the basis of at least the updated probability of confidence associated with each new current detected scene. The output of the filtering processing operation successively delivers filtered detected scenes. [0017]-[0021], [0017] FIGS. 1 to 5 schematically illustrate various modes of implementation and embodiments of the invention. [0066]-[0068], [0100]-[0102])
1. and at least one processor in communication with the at least one storage device, wherein when executing the executable instructions, the at least one processor is configured to cause the system to perform operations including: / 13. implemented on a computing device having at least one processor and at least one storage device, the method comprising: / 20. wherein when executed by at least one processor of a computing device, the at least one set of instructions direct the at least one processor to perform operations including: (Folliot: [0017] FIGS. 1 to 5 schematically illustrate various modes of implementation and embodiments of the invention. [0066]-[0068], [0100]-[0102], [0100] The device APP also includes a processor MDC that is configured to update the initial probability of confidence. [0101] The processor MDC comprises a counter MDC1, the role of which will be detailed below, and an arithmetic logic unit (ALU) MDC2 configured to carry out arithmetic operations and comparisons. [0102] The operations carried out by the arithmetic logic unit MDC2 will be detailed in FIGS. 4 and 5.)
1. obtaining an image captured by a capture device, / 13. obtaining an image captured by a capture device, / 20. obtaining an image captured by a capture device, (Folliot: [0020] According to one aspect, what is proposed is a method for controlling scene detection by a device from among a set of possible reference scenes. [0021] Embodiments provide a method that comprises detection of scenes from among the set of possible reference scenes at successive instants of detection using at least one classification algorithm. Each new current detected scene being assigned an initial probability of confidence. The method also includes updating of the initial probability of confidence depending on a first probability of transition from a previously detected scene to the new current detected scene, and a filtering processing operation on these current detected scenes on the basis of at least the probability of confidence associated with each new current detected scene, the output of the filtering processing operation successively delivering filtered detected scenes.)
1. multiple historical images captured by the capture device before the image, and multiple scene categories; / 13. multiple historical images captured by the capture device before the image, and multiple scene categories; / 20. multiple historical images captured by the capture device before the image, and multiple scene categories; (Folliot: [0021] Embodiments provide a method that comprises detection of scenes from among the set of possible reference scenes at successive instants of detection using at least one classification algorithm. Each new current detected scene being assigned an initial probability of confidence. The method also includes updating of the initial probability of confidence depending on a first probability of transition from a previously detected scene to the new current detected scene, and a filtering processing operation on these current detected scenes on the basis of at least the probability of confidence associated with each new current detected scene, the output of the filtering processing operation successively delivering filtered detected scenes. [0088] The discrimination algorithm implemented in software form in the scene detector MDET is in this case a decision tree that has carried out a learning phase on a measurement database of the environmental sensors. Such a decision tree is particularly easy to implement and requires only a few kilobytes of memory and an operating frequency of less than 0.01 MHz. [0089] It is stored in a program memory MM1. [0090] The decision tree ALC acts on a vector of attributes. The tree comprises a series of nodes. Each node is assigned to test an attribute. [0091] Two branches leave a node. [0092] Moreover, the output of the tree includes leaves corresponding to reference scenes that the device APP is supposed to detect. [0093] These reference scenes may be for example, without limitation, ‘BOAT’, ‘PLANE’, ‘VEHICLE’, ‘WALKING’ scenes that are representative for example, of the environment in which the device APP, in this case the telephone, may be located. [0094] The detector MDET also includes an acquisition circuit ACQ that is configured to acquire current values of the attributes on the basis of the measurement data originating from the sensors. [0098])
1. for each of the multiple scene categories, generating a confidence level of the image belonging to the scene category, the scene category corresponding to a confidence level threshold; / 13. for each of the multiple scene categories, generating a confidence level of the image belonging to the scene category, the scene category corresponding to a confidence level threshold; / 20. for each of the multiple scene categories, generating a confidence level of the image belonging to the scene category, the scene category corresponding to a confidence level threshold; (Folliot: [0099] These two algorithms are also configured to deliver an initial probability of confidence for each detected scene. [0100] The device APP also includes a processor MDC that is configured to update the initial probability of confidence. [0101] The processor MDC comprises a counter MDC1, the role of which will be detailed below, and an arithmetic logic unit (ALU) MDC2 configured to carry out arithmetic operations and comparisons.)
1. for each of the multiple historical images, obtaining an initial scene category to which the historical image belongs; / 13. for each of the multiple historical images, obtaining an initial scene category to which the historical image belongs; / 20. for each of the multiple historical images, obtaining an initial scene category to which the historical image belongs; (Folliot: [0015] For example, one embodiment provides a method for controlling scene detection by a device from among a set of possible reference scenes. The method comprises detection of scenes from among the set of possible reference scenes at successive instants of detection using at least one classification algorithm. Each new current detected scene is assigned an initial probability of confidence. The initial probability of confidence is updated depending on a first probability of transition from a previously detected scene to the new current detected scene. A filtering processing operation is performed on these current detected scenes on the basis of at least the updated probability of confidence associated with each new current detected scene. The output of the filtering processing operation successively delivers filtered detected scenes. [0044] This processing operation of updating the initial probabilities of confidence is in particular compatible, when there is provision to assign an identifier to each reference scene, with a filtering processing operation on the current detected scenes carried out not only on the basis of the updated probability of confidence associated with each new current detected scene, but also on the basis of the identifier of each new current detected scene. (Each detected scene effectively has an identifier, since each detected scene is one of the reference scenes that each has an identifier.))
1. determining, based on initial scene categories of the multiple historical images, a target scene category to which the multiple historical images belong; / 13. determining, based on initial scene categories of the multiple historical images, a target scene category to which the multiple historical images belong; / 20. determining, based on initial scene categories of the multiple historical images, a target scene category to which the multiple historical images belong; (Folliot: [0040] The classification algorithm may deliver the initial probability of confidence associated with the new current detected scene. [0041] If the classification algorithm does not deliver the initial probability of confidence associated with the new current detected scene, it is then possible to assign the new current detected scene a probability of confidence having an arbitrary value. [0047], [0051] For a classification algorithm that is configured to deliver, at each instant of detection, a single detected scene, provision may be made for a register, for example, a shift register, of size M (1×M) in order to form the window of size M. [0052] Thus, in particular, there may be provision for a storage circuit including a shift register of size M forming the window of size M, and, for each new current detected scene, its identifier and the associated updated probability of confidence, which is possibly normalized, is stored in the register, the filtering processing operation is carried out using the M identifiers that are present in the register and their associated updated probability of confidence, which is possibly normalized, and one of the possible scenes, that is to say one of the reference scenes, is delivered as filtered detected scene. [0053] When the content of the register is shifted at each new current detected scene, the identifier that is extracted therefrom is the oldest identifier in terms of time. Of course, however, the identifier that will be delivered after filtering may be any identifier from the register, for example, that of the scene that has just been detected or else that of a preceding scene.)
1. determining multiple updated confidence level thresholds by updating, based on the target scene category, at least a portion of multiple confidence level thresholds corresponding to the multiple scene categories; / 13. determining multiple updated confidence level thresholds by updating, based on the target scene category, at least a portion of multiple confidence level thresholds corresponding to the multiple scene categories; / 20. determining multiple updated confidence level thresholds by updating, based on the target scene category, at least a portion of multiple confidence level thresholds corresponding to the multiple scene categories; (Folliot: [0055] This may also be the case for a neural network that uses the various scenes and their corresponding probability of confidence that are respectively associated with the various neurons of the output layer. [0056] In the case of a classification algorithm (for example, a meta-classifier or a neural network) configured to deliver, at each instant of detection, a group of D scenes, where D is greater than 1, it is possible to provide, in order to form the filtering window of size M, a memory of larger size that is capable of storing a matrix of size D×M (D being the number of rows and M the number of columns). [0057] It is then possible to store, in the storage circuit forming the window of size M, for each new group of D current detected scenes, the updated probabilities of confidence, which are possibly normalized, associated with the identifiers of these D current detected scenes, the filtering processing operation is carried out using the D×M identifiers and their associated updated probability of confidence, which is possibly normalized, and one of the possible scenes, that is to say one of the reference scenes, is delivered as filtered detected scene.)
1. determining, based on the multiple updated confidence level thresholds and confidence levels of the image corresponding to the multiple scene categories, a final scene category to which the image belongs; / 13. determining, based on the multiple updated confidence level thresholds and confidence levels of the image corresponding to the multiple scene categories, a final scene category to which the image belongs; / 20. determining, based on the multiple updated confidence level thresholds and confidence levels of the image corresponding to the multiple scene categories, a final scene category to which the image belongs; (Folliot: [0058] The filtering processing operation may comprise, in particular when the storage circuit comprises a shift register of size M, definition of an integer J greater than or equal to two and less than or equal to the whole part of M/2, and, for each identifier present in the register framed by 2J identical identifiers. The filtering processing operation may also comprise comparison of this framed identifier with the 2J framing identifiers, and, in the event of non-identity of values, replacement of the framed identifier with one of the framing identifiers, and assignment, to the framing identifier replacing the framed identifier, of an updated probability of confidence, which is possibly normalized, calculated on the basis of the updated probabilities of confidence, which are possibly normalized, of the 2J framing identifiers, for example, the average of the probabilities of confidence, which are possibly normalized, of the 2J framing identifiers. [0059] This makes it possible for example, to eliminate isolated detection errors. [0060] When the storage circuit is suited to a classification algorithm that is configured to successively deliver groups of D detected scenes, the filtering processing operation that has just been outlined above in particular to eliminate isolated errors is then advantageously applied to the matrix D×M, row by row.[0061] According to one mode of implementation that is compatible regardless of the configuration of the classification algorithm and therefore regardless of the size of the storage circuit, the filtering processing operation comprises, for each identifier taken into account more than once in the storage circuit, summing of the updated probabilities of confidence, which are possibly normalized, that are associated therewith, the filtered detected scene then being the one whose identifier taken into account in the storage circuit has the highest total updated probability of confidence, which is possibly normalized. [0062]-[0064], [0065] If, by contrast, the ratio is higher than or equal to this threshold, which may be caused by a small variance, the degree of confidence is high, and it may then be decided to deliver the scene that effectively has the highest total updated probability of confidence, which is possibly normalized, as detected scene at the output of the filter.)
1. and processing the image based on the final scene category of the image. / 13. and processing the image based on the final scene category of the image. / 20. and processing the image based on the final scene category of the image. (Folliot: [0062] According to one possible more elaborate variant, which is also compatible regardless of the size of the storage circuit and that makes it possible in particular to provide an indication as to the variability of the filter, the filtering processing operation comprises, for each identifier taken into account more than once in the storage circuit, summing of the updated probabilities of confidence, which are possibly normalized, that are associated therewith, formulation of a probability density function for the identifiers that is centered on the identifier having the highest total updated probability of confidence, which is possibly normalized, calculation of the variance of this function, calculation of a ratio between the highest total probability of confidence, which is possibly normalized, and the variance, and comparison of this ratio with a threshold and selection of the filtered detected scene depending on the result of the comparison. [0063]-[0065], [0066] According to another aspect, there is also proposed a device comprising a detector configured to detect scenes from among a set of possible reference scenes at successive instants of detection using at least one classification algorithm, each new current detected scene being assigned an initial probability of confidence. A processor is configured to update the initial probability of confidence depending on a first probability of transition from a previously detected scene to the new current detected scene. A filter is configured to perform a filtering processing operation on the basis of at least the updated probability of confidence associated with each new current detected scene and to successively deliver filtered detected scenes. [0074] According to one embodiment, the classification algorithm is configured to deliver the probability of confidence associated with the new current detected scene. [0075] According to one embodiment, if the classification algorithm is not configured to deliver the probability of confidence associated with the new current detected scene, a probability of confidence having an arbitrary value is assigned to the new current detected scene)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claims 1-8 and 13-20 are rejected under 35 U.S.C. 103 as being unpatentable over Folliot et al. (US PGPub US 20190287009 A1, hereby referred to as “Folliot”), in view of Hatfield et al. (US PGPub US 2025/0150511 A1, filed on November 2, 2023), hereby referred to as “Hatfield”.
Consider Claims 1, 13 and 20.
Folliot teaches:
1. A system, comprising: at least one storage device storing executable instructions; / 13. A method / 20. A non-transitory computer readable medium, comprising at least one set of instructions, (Folliot: abstract, An embodiment provides a method for controlling scene detection by a device from among a set of possible reference scenes. The method includes detection of scenes from among the set of possible reference scenes at successive instants of detection using at least one classification algorithm. Each new current detected scene is assigned an initial probability of confidence. The initial probability of confidence is updated depending on a first probability of transition from a previously detected scene to the new current detected scene. A filtering processing operation is performed on these current detected scenes on the basis of at least the updated probability of confidence associated with each new current detected scene. The output of the filtering processing operation successively delivers filtered detected scenes. [0017]-[0021], [0017] FIGS. 1 to 5 schematically illustrate various modes of implementation and embodiments of the invention. [0066]-[0068], [0100]-[0102])
1. and at least one processor in communication with the at least one storage device, wherein when executing the executable instructions, the at least one processor is configured to cause the system to perform operations including: / 13. implemented on a computing device having at least one processor and at least one storage device, the method comprising: / 20. wherein when executed by at least one processor of a computing device, the at least one set of instructions direct the at least one processor to perform operations including: (Folliot: [0017] FIGS. 1 to 5 schematically illustrate various modes of implementation and embodiments of the invention. [0066]-[0068], [0100]-[0102], [0100] The device APP also includes a processor MDC that is configured to update the initial probability of confidence. [0101] The processor MDC comprises a counter MDC1, the role of which will be detailed below, and an arithmetic logic unit (ALU) MDC2 configured to carry out arithmetic operations and comparisons. [0102] The operations carried out by the arithmetic logic unit MDC2 will be detailed in FIGS. 4 and 5.)
1. obtaining an image captured by a capture device, / 13. obtaining an image captured by a capture device, / 20. obtaining an image captured by a capture device, (Folliot: [0020] According to one aspect, what is proposed is a method for controlling scene detection by a device from among a set of possible reference scenes. [0021] Embodiments provide a method that comprises detection of scenes from among the set of possible reference scenes at successive instants of detection using at least one classification algorithm. Each new current detected scene being assigned an initial probability of confidence. The method also includes updating of the initial probability of confidence depending on a first probability of transition from a previously detected scene to the new current detected scene, and a filtering processing operation on these current detected scenes on the basis of at least the probability of confidence associated with each new current detected scene, the output of the filtering processing operation successively delivering filtered detected scenes.)
1. multiple historical images captured by the capture device before the image, and multiple scene categories; / 13. multiple historical images captured by the capture device before the image, and multiple scene categories; / 20. multiple historical images captured by the capture device before the image, and multiple scene categories; (Folliot: [0021] Embodiments provide a method that comprises detection of scenes from among the set of possible reference scenes at successive instants of detection using at least one classification algorithm. Each new current detected scene being assigned an initial probability of confidence. The method also includes updating of the initial probability of confidence depending on a first probability of transition from a previously detected scene to the new current detected scene, and a filtering processing operation on these current detected scenes on the basis of at least the probability of confidence associated with each new current detected scene, the output of the filtering processing operation successively delivering filtered detected scenes. [0088] The discrimination algorithm implemented in software form in the scene detector MDET is in this case a decision tree that has carried out a learning phase on a measurement database of the environmental sensors. Such a decision tree is particularly easy to implement and requires only a few kilobytes of memory and an operating frequency of less than 0.01 MHz. [0089] It is stored in a program memory MM1. [0090] The decision tree ALC acts on a vector of attributes. The tree comprises a series of nodes. Each node is assigned to test an attribute. [0091] Two branches leave a node. [0092] Moreover, the output of the tree includes leaves corresponding to reference scenes that the device APP is supposed to detect. [0093] These reference scenes may be for example, without limitation, ‘BOAT’, ‘PLANE’, ‘VEHICLE’, ‘WALKING’ scenes that are representative for example, of the environment in which the device APP, in this case the telephone, may be located. [0094] The detector MDET also includes an acquisition circuit ACQ that is configured to acquire current values of the attributes on the basis of the measurement data originating from the sensors. [0098])
1. for each of the multiple scene categories, generating a confidence level of the image belonging to the scene category, the scene category corresponding to a confidence level threshold; / 13. for each of the multiple scene categories, generating a confidence level of the image belonging to the scene category, the scene category corresponding to a confidence level threshold; / 20. for each of the multiple scene categories, generating a confidence level of the image belonging to the scene category, the scene category corresponding to a confidence level threshold; (Folliot: [0099] These two algorithms are also configured to deliver an initial probability of confidence for each detected scene. [0100] The device APP also includes a processor MDC that is configured to update the initial probability of confidence. [0101] The processor MDC comprises a counter MDC1, the role of which will be detailed below, and an arithmetic logic unit (ALU) MDC2 configured to carry out arithmetic operations and comparisons.)
1. for each of the multiple historical images, obtaining an initial scene category to which the historical image belongs; / 13. for each of the multiple historical images, obtaining an initial scene category to which the historical image belongs; / 20. for each of the multiple historical images, obtaining an initial scene category to which the historical image belongs; (Folliot: [0015] For example, one embodiment provides a method for controlling scene detection by a device from among a set of possible reference scenes. The method comprises detection of scenes from among the set of possible reference scenes at successive instants of detection using at least one classification algorithm. Each new current detected scene is assigned an initial probability of confidence. The initial probability of confidence is updated depending on a first probability of transition from a previously detected scene to the new current detected scene. A filtering processing operation is performed on these current detected scenes on the basis of at least the updated probability of confidence associated with each new current detected scene. The output of the filtering processing operation successively delivers filtered detected scenes. [0044] This processing operation of updating the initial probabilities of confidence is in particular compatible, when there is provision to assign an identifier to each reference scene, with a filtering processing operation on the current detected scenes carried out not only on the basis of the updated probability of confidence associated with each new current detected scene, but also on the basis of the identifier of each new current detected scene. (Each detected scene effectively has an identifier, since each detected scene is one of the reference scenes that each has an identifier.))
1. determining, based on initial scene categories of the multiple historical images, a target scene category to which the multiple historical images belong; / 13. determining, based on initial scene categories of the multiple historical images, a target scene category to which the multiple historical images belong; / 20. determining, based on initial scene categories of the multiple historical images, a target scene category to which the multiple historical images belong; (Folliot: [0040] The classification algorithm may deliver the initial probability of confidence associated with the new current detected scene. [0041] If the classification algorithm does not deliver the initial probability of confidence associated with the new current detected scene, it is then possible to assign the new current detected scene a probability of confidence having an arbitrary value. [0047], [0051] For a classification algorithm that is configured to deliver, at each instant of detection, a single detected scene, provision may be made for a register, for example, a shift register, of size M (1×M) in order to form the window of size M. [0052] Thus, in particular, there may be provision for a storage circuit including a shift register of size M forming the window of size M, and, for each new current detected scene, its identifier and the associated updated probability of confidence, which is possibly normalized, is stored in the register, the filtering processing operation is carried out using the M identifiers that are present in the register and their associated updated probability of confidence, which is possibly normalized, and one of the possible scenes, that is to say one of the reference scenes, is delivered as filtered detected scene. [0053] When the content of the register is shifted at each new current detected scene, the identifier that is extracted therefrom is the oldest identifier in terms of time. Of course, however, the identifier that will be delivered after filtering may be any identifier from the register, for example, that of the scene that has just been detected or else that of a preceding scene.)
1. determining multiple updated confidence level thresholds by updating, based on the target scene category, at least a portion of multiple confidence level thresholds corresponding to the multiple scene categories; / 13. determining multiple updated confidence level thresholds by updating, based on the target scene category, at least a portion of multiple confidence level thresholds corresponding to the multiple scene categories; / 20. determining multiple updated confidence level thresholds by updating, based on the target scene category, at least a portion of multiple confidence level thresholds corresponding to the multiple scene categories; (Folliot: [0055] This may also be the case for a neural network that uses the various scenes and their corresponding probability of confidence that are respectively associated with the various neurons of the output layer. [0056] In the case of a classification algorithm (for example, a meta-classifier or a neural network) configured to deliver, at each instant of detection, a group of D scenes, where D is greater than 1, it is possible to provide, in order to form the filtering window of size M, a memory of larger size that is capable of storing a matrix of size D×M (D being the number of rows and M the number of columns). [0057] It is then possible to store, in the storage circuit forming the window of size M, for each new group of D current detected scenes, the updated probabilities of confidence, which are possibly normalized, associated with the identifiers of these D current detected scenes, the filtering processing operation is carried out using the D×M identifiers and their associated updated probability of confidence, which is possibly normalized, and one of the possible scenes, that is to say one of the reference scenes, is delivered as filtered detected scene.)
1. determining, based on the multiple updated confidence level thresholds and confidence levels of the image corresponding to the multiple scene categories, a final scene category to which the image belongs; / 13. determining, based on the multiple updated confidence level thresholds and confidence levels of the image corresponding to the multiple scene categories, a final scene category to which the image belongs; / 20. determining, based on the multiple updated confidence level thresholds and confidence levels of the image corresponding to the multiple scene categories, a final scene category to which the image belongs; (Folliot: [0058] The filtering processing operation may comprise, in particular when the storage circuit comprises a shift register of size M, definition of an integer J greater than or equal to two and less than or equal to the whole part of M/2, and, for each identifier present in the register framed by 2J identical identifiers. The filtering processing operation may also comprise comparison of this framed identifier with the 2J framing identifiers, and, in the event of non-identity of values, replacement of the framed identifier with one of the framing identifiers, and assignment, to the framing identifier replacing the framed identifier, of an updated probability of confidence, which is possibly normalized, calculated on the basis of the updated probabilities of confidence, which are possibly normalized, of the 2J framing identifiers, for example, the average of the probabilities of confidence, which are possibly normalized, of the 2J framing identifiers. [0059] This makes it possible for example, to eliminate isolated detection errors. [0060] When the storage circuit is suited to a classification algorithm that is configured to successively deliver groups of D detected scenes, the filtering processing operation that has just been outlined above in particular to eliminate isolated errors is then advantageously applied to the matrix D×M, row by row.[0061] According to one mode of implementation that is compatible regardless of the configuration of the classification algorithm and therefore regardless of the size of the storage circuit, the filtering processing operation comprises, for each identifier taken into account more than once in the storage circuit, summing of the updated probabilities of confidence, which are possibly normalized, that are associated therewith, the filtered detected scene then being the one whose identifier taken into account in the storage circuit has the highest total updated probability of confidence, which is possibly normalized. [0062]-[0064], [0065] If, by contrast, the ratio is higher than or equal to this threshold, which may be caused by a small variance, the degree of confidence is high, and it may then be decided to deliver the scene that effectively has the highest total updated probability of confidence, which is possibly normalized, as detected scene at the output of the filter.)
1. and processing the image based on the final scene category of the image. / 13. and processing the image based on the final scene category of the image. / 20. and processing the image based on the final scene category of the image. (Folliot: [0062] According to one possible more elaborate variant, which is also compatible regardless of the size of the storage circuit and that makes it possible in particular to provide an indication as to the variability of the filter, the filtering processing operation comprises, for each identifier taken into account more than once in the storage circuit, summing of the updated probabilities of confidence, which are possibly normalized, that are associated therewith, formulation of a probability density function for the identifiers that is centered on the identifier having the highest total updated probability of confidence, which is possibly normalized, calculation of the variance of this function, calculation of a ratio between the highest total probability of confidence, which is possibly normalized, and the variance, and comparison of this ratio with a threshold and selection of the filtered detected scene depending on the result of the comparison. [0063]-[0065], [0066] According to another aspect, there is also proposed a device comprising a detector configured to detect scenes from among a set of possible reference scenes at successive instants of detection using at least one classification algorithm, each new current detected scene being assigned an initial probability of confidence. A processor is configured to update the initial probability of confidence depending on a first probability of transition from a previously detected scene to the new current detected scene. A filter is configured to perform a filtering processing operation on the basis of at least the updated probability of confidence associated with each new current detected scene and to successively deliver filtered detected scenes. [0074] According to one embodiment, the classification algorithm is configured to deliver the probability of confidence associated with the new current detected scene. [0075] According to one embodiment, if the classification algorithm is not configured to deliver the probability of confidence associated with the new current detected scene, a probability of confidence having an arbitrary value is assigned to the new current detected scene)
Even if Folliot does not specifically teach: “initial scene categories of the multiple historical images”
Hatfield teaches:
1. A system, comprising: at least one storage device storing executable instructions; / 13. A method / 20. A non-transitory computer readable medium, comprising at least one set of instructions, (Hatfield: abstract, Methods, computer program products, and systems are presented. The method computer program products, and systems can include, for instance: with a user engaged in a user session within a first environment, examining historical data of the user, wherein the examining historical data of the user includes examining historical data of a previously recognized article of the user previously recognized in a second environment; and provisioning an article of the user within the first environment to include a functional behavior. Figure 1, [0024] System 100 for use in provisioning articles including electronic device articles is shown in FIG. 1 . System 100 can include manager system 110 having an associated data repository 108, user equipment (UE) devices 130A-130Z, and social media system 140. Manager system 110, UE devices 130A-130Z and social media system 140 can be in communication with one another via network 190. Network 190 can include a physical network and/or a virtual network. A physical network can include for example, a physical telecommunications network connecting numerous computing nodes or systems, such as computer servers and computer clients. A virtual network can, for example, combine numerous physical networks or parts thereof into a logical virtual network. In another example, numerous virtual networks can be defined over a single physical network. In the context of UE devices 130A-130Z, “Z” can refer to an integer of any arbitrary value.)
1. and at least one processor in communication with the at least one storage device, wherein when executing the executable instructions, the at least one processor is configured to cause the system to perform operations including: / 13. implemented on a computing device having at least one processor and at least one storage device, the method comprising: / 20. wherein when executed by at least one processor of a computing device, the at least one set of instructions direct the at least one processor to perform operations including: (Hatfield: Figure 6, [0146] Referring now to FIG. 6B a block diagram of the data flow 400 between various hardware and software components of a client/server environment implementing a virtual world is shown according to an illustrative embodiment. Virtual World client application 410 is a software application, executing on a client data processing system. Virtual World client application 410 allows for connection to, and interaction within a virtual world. In response to an indication from a user, virtual world client application 410 sends snapshot request 412 to virtual world server 414. Snapshot request 412 is a request for a graphical rendering of the viewable field of a particular user's avatar, as displayed by virtual world client application 410. The snapshot request can be a request for a static image, or a video of the rendering of the viewable field. Virtual world server 414 is a data processing system that executes software to enable a virtual world, such as virtual world server 300. Responsive to receiving snapshot request 412, virtual world server 414 forwards scene descriptions 416 to backend rendering system 418. Scene description 416 is a non-graphical representation of the viewable field of a particular user's avatar. Scene description 416 includes virtual world location information such as an exact camera location within the virtual world, and a camera angle vector. The exact camera location within the virtual world and the camera angle vector can be expressed as coordinates in a 3D coordinate system, such as for example in an <X, Y, Z> format, Scene description 416 can also include information regarding the format and delivery of the rendered snapshot, such as for example but not limited to, an image size, a file type format, delivery options, whether to include avatars within the snapshot, and desired fidelity/resolution of the snapshot. Scene description 416 going to backend rendering system 418 are a subset of the updates going to the client. Backend rendering system 418 is a graphics-processing unit capable of generating images and video of activity within a virtual world. Backend rendering system 418 is capable of rendering images and video of higher quality than local 3D rendering hardware and software running virtual world client application 410 because backend rendering system 418 utilizes images and textures that are themselves not used to create the virtual world as rendered by virtual world client application 410. Backend rendering system 418 produces high quality images by using higher quality models, shaders, and textures, or by implementing a ray tracing system. The models, shaders, textures, and ray tracing system utilized by backend rendering system 418 are not used in rendering the virtual world as is typically rendered by virtual world client application 410. In one illustrative embodiment, backend rendering system 418 implements high quality ray tracing effects using a Cell Broadband Engine Architecture based system. In one illustrative embodiment, backend rendering system 418 implements high quality models, shaders, and textures using traditional graphical processing unit hardware. Responsive to receiving scene description 416, backend rendering system 418 sends request 420 for high fidelity objects to visual database 422. Visual database 422 contains textures, geometry, and shaders that are needed to implement the virtual world. A scene description is downloaded onto backend rendering system 418, and the needed textures, geometry, and shaders are downloaded from the Visual database. Updates are then sent to backend rendering system 418 as the avatar moves in the scene. Backend rendering system 418 adjusts and moves the rendered avatars and objects accordingly. Visual database 422 stores geometric data, textures, and effects data associated with the form or shape of avatars and objects in the virtual world. Visual database 422 can be referenced by an Object avatar-rendering table in order to obtain geometric data, textures, and effects data to render the virtual world. Visual database 422 contains lightweight graphics 424 and high fidelity graphics 426. Lightweight graphics 424 include the 3D geometry, textures, shaders that are utilized by both virtual world server 414 and virtual world client application 410 to render the virtual world to the client. Lightweight graphics 424 are delivered to the virtual world server 414 in response to a request 428 for low fidelity objects sent from virtual world server 414 to visual database 422. High fidelity graphics 426 is a high fidelity version of geometry, textures, shaders that are utilized by backend rendering system 418 to create high fidelity snapshots and videos. Due to the bandwidth and processing requirements, high fidelity graphics 426 are not utilized by virtual world client application 410 in rendering the virtual world in real time. Backend rendering system 418 utilizes high fidelity graphics 426 to create high fidelity video and images 430. In one illustrative embodiment, backend rendering system 418 implements high quality ray tracing effects using a cell based system. In one illustrative embodiment, backend rendering System 418 implements high quality models, shaders, textures using traditional graphical processing unit hardware. In one illustrative embodiment, backend rendering system 418 implements high quality ray tracing effects using the Cell Broadband Engine Architecture based system. Backend rendering system 418 ray tracing determines the visibility of surfaces within an image by tracing imaginary rays of light from viewer's eye to the object. A separate ray is utilized for each resolved point, typically for each pixel of the display. For example, for a modest image resolution of 1064×768, a ray-tracer needs to trace 1064×768, or 817,152 separate rays for each displayed image. While the ray-traced image is very realistic due to the separate casting of each resolved point, the sheer number of rays that must be traced results in longer process times that are typically unsuitable for real time processing that must occur when implementing a virtual world. In one illustrative embodiment, backend rendering system 418 implements high quality ray tracing effects using a cell based system. A cell based system is a data processing system using the Cell Broadband Engine Architecture. The Cell Broadband Engine Architecture is a multi-core processor. The cell-based, system differs from traditional multi-core processors in that the cell based system does not consist of multiple copies of the same core. Instead, the cell based system utilizes heterogeneous processors, consisting of one PowerPC core and a plurality of synergistic co-processor elements. The synergistic co-processor elements are typically smaller than traditional cores and are designed for streaming workloads. The PowerPC core performs synchronization tasks and executes non-parallizeable code. The synergistic co-processor elements perform the balance of the work in a streaming manner. In one illustrative embodiment, backend rendering system 418 creates create high fidelity video and images 430 by sending scene updates to the backend rendering system as the client moves the avatar moves in the scene and objects move or change in the scene. The backend rendering system adjusts and moves the viewing position and rendered avatars and objects accordingly. In one illustrative embodiment, ray-traced images rendered by the Cell Broadband Engine Architecture based system of backend rendering system 418 are created at fast time intervals using scene updates. To create the illusion of smoother movement between the ray-traced key frames, the key frame animation software extrapolates in-between geometry and positions from scene updates to create a smoother animation. As the client navigates the virtual world, avatar and camera movement 432 are communicated to virtual world server 414. Additional scene descriptions 434 are passed to backend rendering system 418 and stored in scene update database 436 until additional scene descriptions 434 are utilized by backend rendering system 418. Scene update database 436 is a storage that holds scene descriptions, such as additional scene descriptions 434, until those scene descriptions are utilized by backend rendering system 418 to create high fidelity video and images 430. Thus, scene update database 436 acts as a buffer system, temporarily storing additional scene descriptions 434 for the backend rendering system 418. In this manner, backend rendering system 418 is able to function in a non-real time. Backend rendering system 418 is therefore able to create realistic images and videos without sacrificing resolution so that processing can be performed in real time. Once backend rendering system 418 creates high fidelity video and images 430, high fidelity video and images 430 are stored in video database 438 until high fidelity video and images 430 is delivered to a client system.)
1. obtaining an image captured by a capture device, / 13. obtaining an image captured by a capture device, / 20. obtaining an image captured by a capture device, (Hatfield: [0024] System 100 for use in provisioning articles including electronic device articles is shown in FIG. 1 . System 100 can include manager system 110 having an associated data repository 108, user equipment (UE) devices 130A-130Z, and social media system 140. Manager system 110, UE devices 130A-130Z and social media system 140 can be in communication with one another via network 190. Network 190 can include a physical network and/or a virtual network. A physical network can include for example, a physical telecommunications network connecting numerous computing nodes or systems, such as computer servers and computer clients. A virtual network can, for example, combine numerous physical networks or parts thereof into a logical virtual network. In another example, numerous virtual networks can be defined over a single physical network. In the context of UE devices 130A-130Z, “Z” can refer to an integer of any arbitrary value.[0038]-[0044], [0045] In one embodiment, manager system 110 running recognition process 116 can include manager system 110 examining spatial image data representing a feature of interest. Manager system 110 running recognition process 116 can include manager system employing pattern recognition processing using one or more of, e.g., feature extraction algorithms, classification algorithms, and/or clustering algorithms. In one embodiment, manager system 110 running recognition process 114 can include performing of digital image processing. Digital image processing can include, e.g., filtering, edge detection, shape classification, optical character recognition (OCR), and/or encoded information decoding.)
1. multiple historical images captured by the capture device before the image, and multiple scene categories; / 13. multiple historical images captured by the capture device before the image, and multiple scene categories; / 20. multiple historical images captured by the capture device before the image, and multiple scene categories; (Hatfield: [0076]-[0077] With further reference to scoring formula A, manager system 110 at generating block 1110 can assign scoring values under factor FV4 in dependence on an output resulting from query of provisioning data predictive model 4102 as shown in FIG. 4A. Provisioning data predictive model 4102 can predict a functional behavior provisioning data action of a user specifying a functional behavior for provisioning with respect to a detected virtual world article. Provisioning data predictive model 4102 can be trained with historical session data associated to a current user as well as a crowdsourced collection of other users of system 100 and manager system 110. Provisioning data predictive model 4102 can be trained with historical data of historical virtual reality sessions of a current user as well as a crowdsourced collection of all other users of system 100 and manager system 110. Provisioning data predictive model 4102 can be trained with training data and once trained can be responsive to query data for return of predictions as to what actions a user will take with respective provisioning a detected virtual world article. [0079] The article classifier for training predictive model 4102 can be a classifier for an article during an historical virtual reality session in which a user was prompted for prompting, e.g., smartphone as described with reference in the example of FIG. 3A. The article usage classifier for training predictive model 4102 can refer to the usage of the detected article during the virtual reality session, e.g., holding, browsing, telephone calling, texting, and the like. The article provisioning data action classifier for training predictive model 4102 can refer to the functional behavior provisioning of the historical article by historical user and in an historical session mediated by manager system 110. The accompanying articles vector for training predictive model 4102 can specify identifiers for other articles in an environment of the article, the attendees classifier can classify for attendees in the environment of the virtual user during an historical virtual reality session. Possible attendees classifiers can be, e.g., no other attendees, one other attendee friend, one other attendee stranger, multiple other attendees friends, multiple other attendees strangers, and so on. Environmental classifiers can include, e.g., outside, residence, work environment, article acquisition venue and the like. The user classifier for training provisioning data predictive model 4102 can be based on clustering analysis, wherein manager system 110 identifies clusters of similar users based on preferences of such users of system 100 and manager system 110. [0080] Clustering analysis for use in determining similar users is set forth in reference to FIGS. 5A-5B.)
1. for each of the multiple scene categories, generating a confidence level of the image belonging to the scene category, the scene category corresponding to a confidence level threshold; / 13. for each of the multiple scene categories, generating a confidence level of the image belonging to the scene category, the scene category corresponding to a confidence level threshold; / 20. for each of the multiple scene categories, generating a confidence level of the image belonging to the scene category, the scene category corresponding to a confidence level threshold; (Hatfield: [0081] The user classifier for training predictive model 4102 set forth in FIG. 2 can refer to a type classification of a given user. Manager system 110 can establish a user classification for any user of system 100 by application of clustering analysis as set forth in FIGS. 5A-5B, wherein identified user clusters define user classifications. [0082]-[0083] FIG. 5A depicts a clustering machine learning model 5102 and FIG. 5B depicts functioning of the machine learning model. FIG. 5B depicts plotted dimensional data points for multiple users of system 100 for a designated key topic. For the key topic, manager system 110 can assign a topic strength value and a topic sentiment value. Manager system 110 can apply topic strengths as indicated in FIG. 5B in dependence on account of times the key topic is extracted per predetermined volume of asset data, e.g., based on a count of topic extractions for every 1000 words. Manager system 110 can apply a topic sentiment by performance of sentiment extraction using natural language processing on text based content associated to respective key topic extractions. Manager system 110 can identify clusters of users by their close proximity in Euclidean space, and can assign the clusters various cluster identifiers, e.g., cluster 1, cluster 2, and cluster 3 as depicted in FIG. 5B. Referring to FIG. 5B, manager system 110 can assign sentiment values to users for the key topic depicted on a scale, e.g., of 0.0 to 1.0 with values from 0.0 to 0.5 being negative sentiment values, the value of 0.5 being a neutral sentiment and the values 0.5 to 1.0 being positive sentiment values. While a single key topic is depicted in FIG. 5B, manager system 110 can expand the clustering analysis across additional dimensions defined by additional key topics. [0084] Predictive model 4102, once trained, can be capable of returning predictions. For example, with reference to FIG. 4A, provisioning data predictive model 4102 can be queried with query data that specifies an article classifier, an article usage classifier, an accompanying articles vector, an attendees classifier, an environmental classifier, and a user classifier for the current virtual reality session for which prompting data is generated at block 1110. In response to the query data, provisioning data predictive model 4102 can generate a predicted provisioning action of a user, e.g., whether the user based on historical sessions of other users in the current user will provision a virtual current article, e.g., smartphone to assume new behavior, e.g., new functional behavior, e.g., contact list functionality, e.g., with auto contacting phone functionality, texting functionality, browsing functionality, picture taking functionality, and the like. Trained as described, provisioning predictive model 4102 is able to predict provisioning actions of a user. [0085] Manager system 110 provisioning data predictive model 4102 in generating predicted provisioning data action decisions can assign confidence levels associated to the various predictions. The confidence level, in one example, can be dependent on trends exhibited by the training data and a volume of training data supporting certain prediction.)
1. for each of the multiple historical images, obtaining an initial scene category to which the historical image belongs; / 13. for each of the multiple historical images, obtaining an initial scene category to which the historical image belongs; / 20. for each of the multiple historical images, obtaining an initial scene category to which the historical image belongs; (Hatfield: [0086] Manager system 110 can scale scoring values under factor FV4 (predictive model factor) by reference to a decision data structure in which there are assigned action intensity values to particularly predicted provisioning actions. Scoring values assigned by manager system 110 under factor FV4 can be assigned according to the decision data structure of Table A, when predicted action confidence level intensity value are all considered in deriving an output scoring value under factor FV4. [0089] Referring to factor FV6, factor FV6 can be a tournament factor. Manager system 110 for assigning scoring values under factor FV6 can subject detected articles detected within a scene to tournament selection evaluating and can scale assigned scoring values to detected articles according to tournament selection scoring.)
1. determining, based on initial scene categories of the multiple historical images, a target scene category to which the multiple historical images belong; / 13. determining, based on initial scene categories of the multiple historical images, a target scene category to which the multiple historical images belong; / 20. determining, based on initial scene categories of the multiple historical images, a target scene category to which the multiple historical images belong; (Hatfield: [0140] In reference to generating 1110, manager system 110 can generate prompting data for prompting provisioning action in response to detection of a virtual world article. In another embodiment, generating at block 1110 can be triggered in dependence on manager system 110 predicting that the user wishes to invoke a certain virtual article, and then manager system 110 can responsively generate prompting data for provisioning that certain virtual article. FIG. 4C depicts an article instantiation predictive model 4302 which can be configured to generate predictions that a user wishes to instantiate a virtual article. Article instantiation predictive model 4302 can be trained with iterations of training data and once trained article instantiation predictive model 4302 is able to predict when a user wishes to instantiate a certain article. Article instantiation predictive model 4302 can be trained with an iteration of training data each time that an historical user of system 100 and manager system 110 uses menu 3118 as shown in FIGS. 3A and 3B. Referring to FIGS. 3A and 3B, menu 3118 can be configured so that when a user accesses menu 3118, a drop-down list of articles for instantiation can appear which is selectable by a user. Manager system 110 can be configured so that each time an historical user uses menu 3118 to instantiate a new virtual article that is not currently rendered within a scene, manager system 110 registers an article instantiation event and uses specifying data of the instantiation of that event as training data for training article instantiation predictive model 4302. An iteration of training data for training article instantiation predictive model 4302 can include an article classifier, i.e., the article selected for instantiation and accompanying articles vector listing identifiers for all articles within the scene currently rendered, e.g., referring to, e.g., in the case of FIG. 3B, the virtual smartphone and the virtual bicycle. Attendees classifier, i.e., in identifiers for attendees of an environment during an instantiation event, an environment classifier for the instantiation event, e.g., in office, in residence, outdoors, etc. and a user classifier as described in connection with user clustering predictive model 5102 as explained in reference to FIGS. 5A and 5B. Trained as described, article instantiation predictive model 4302 is able to learn a relationship between attributes of a current environment of a user and their influence on a user's decision to instantiate a new virtual article into a virtual reality environment. Trained as described, article instantiation predictive model 4302 is able to predict when a user wishes to instantiate a virtual article. Article instantiation predictive model 4302 once trained, is able to respond to query data. Query data for querying article instantiation predictive model 4302 can include, for the current scene, being generated by a virtual reality platform in accompanying articles classifier listing the currently rendered articles of a current environment. An attendees classifier listing an attendees classification for the current environment, an environment classifier for the current environment, e.g., in residence, in business, outdoors and a user classifier as explained with reference to FIGS. 5A and 5B. In response to the described query data, article instantiation predictive model 4302 can return a prediction as to an article that the user currently wishes to instantiate in a current virtual reality scene. In one embodiment, manager system 110 at block 1109, while ascertaining whether an article has been rendered, can be querying article instantiation predictive model 4302 in order to predict whether the user. In one embodiment, manager system 110 can proceed to generating block 1110 on predicting that a user wishes to instantiate a virtual article. In one example, the predicting can be qualified when a confidence level associated to the predicting satisfies a threshold. In addition to or in place of using predictive model 4302 for use in predicting that a user wishes to instantiate a virtual article, manager system 110 can apply the tournament scoring process set forth herein in reference to Eq. 1-Eq. 6 for predicting that a user wishes to instantiate a virtual article.)
1. determining multiple updated confidence level thresholds by updating, based on the target scene category, at least a portion of multiple confidence level thresholds corresponding to the multiple scene categories; / 13. determining multiple updated confidence level thresholds by updating, based on the target scene category, at least a portion of multiple confidence level thresholds corresponding to the multiple scene categories; / 20. determining multiple updated confidence level thresholds by updating, based on the target scene category, at least a portion of multiple confidence level thresholds corresponding to the multiple scene categories; (Hatfield: [0139] It will be seen that in a subsequent iteration of the loop of blocks 2310-2316, manager system 110 at an iteration of block 2310 in the described example described in reference to FIG. 1 can detect the appearance of the physical article IoT bicycle 1302. On the detection of the IoT bicycle 1302, manager system 110 can generate next prompting data 3152 as set forth in FIG. 3D which prompting data can be generated in the manner described in reference to generating block 1119, except this time with respect to the differently detected physical article, namely the IoT bicycle depicted in FIG. 1 . For the different IoT bicycle article, different functional behavior provisioning options can apply, e.g., upload route history, upload maps, upload riding friends, synchronize riding history, and the like. The different functional behavior options can be specified within the variable text “XXXX” defining prompting data 3152, wherein each option is specified by variable prompting data text 3153, 3154, or 3155 at a different elevations depending on the confidence level of the option being selected as determined by querying of predictive model 4102. [0140] In reference to generating 1110, manager system 110 can generate prompting data for prompting provisioning action in response to detection of a virtual world article. In another embodiment, generating at block 1110 can be triggered in dependence on manager system 110 predicting that the user wishes to invoke a certain virtual article, and then manager system 110 can responsively generate prompting data for provisioning that certain virtual article. FIG. 4C depicts an article instantiation predictive model 4302 which can be configured to generate predictions that a user wishes to instantiate a virtual article. Article instantiation predictive model 4302 can be trained with iterations of training data and once trained article instantiation predictive model 4302 is able to predict when a user wishes to instantiate a certain article. Article instantiation predictive model 4302 can be trained with an iteration of training data each time that an historical user of system 100 and manager system 110 uses menu 3118 as shown in FIGS. 3A and 3B.)
1. determining, based on the multiple updated confidence level thresholds and confidence levels of the image corresponding to the multiple scene categories, a final scene category to which the image belongs; / 13. determining, based on the multiple updated confidence level thresholds and confidence levels of the image corresponding to the multiple scene categories, a final scene category to which the image belongs; / 20. determining, based on the multiple updated confidence level thresholds and confidence levels of the image corresponding to the multiple scene categories, a final scene category to which the image belongs; (Hatfield: [0140] Manager system 110 can be configured so that each time an historical user uses menu 3118 to instantiate a new virtual article that is not currently rendered within a scene, manager system 110 registers an article instantiation event and uses specifying data of the instantiation of that event as training data for training article instantiation predictive model 4302. An iteration of training data for training article instantiation predictive model 4302 can include an article classifier, i.e., the article selected for instantiation and accompanying articles vector listing identifiers for all articles within the scene currently rendered, e.g., referring to, e.g., in the case of FIG. 3B, the virtual smartphone and the virtual bicycle. Attendees classifier, i.e., in identifiers for attendees of an environment during an instantiation event, an environment classifier for the instantiation event, e.g., in office, in residence, outdoors, etc. and a user classifier as described in connection with user clustering predictive model 5102 as explained in reference to FIGS. 5A and 5B. Trained as described, article instantiation predictive model 4302 is able to learn a relationship between attributes of a current environment of a user and their influence on a user's decision to instantiate a new virtual article into a virtual reality environment. Trained as described, article instantiation predictive model 4302 is able to predict when a user wishes to instantiate a virtual article. Article instantiation predictive model 4302 once trained, is able to respond to query data. Query data for querying article instantiation predictive model 4302 can include, for the current scene, being generated by a virtual reality platform in accompanying articles classifier listing the currently rendered articles of a current environment. An attendees classifier listing an attendees classification for the current environment, an environment classifier for the current environment, e.g., in residence, in business, outdoors and a user classifier as explained with reference to FIGS. 5A and 5B. In response to the described query data, article instantiation predictive model 4302 can return a prediction as to an article that the user currently wishes to instantiate in a current virtual reality scene. In one embodiment, manager system 110 at block 1109, while ascertaining whether an article has been rendered, can be querying article instantiation predictive model 4302 in order to predict whether the user. In one embodiment, manager system 110 can proceed to generating block 1110 on predicting that a user wishes to instantiate a virtual article. In one example, the predicting can be qualified when a confidence level associated to the predicting satisfies a threshold. In addition to or in place of using predictive model 4302 for use in predicting that a user wishes to instantiate a virtual article, manager system 110 can apply the tournament scoring process set forth herein in reference to Eq. 1-Eq. 6 for predicting that a user wishes to instantiate a virtual article. [0144] Manager system 110 running virtual world process 111 can include manager system 110 operating the system depicted in FIG. 6A. FIG. 6A is a block diagram illustrating a virtual world server in accordance with an illustrative embodiment. Server 300 is a server associated with a virtual world. Server 300 may be a single, stand-alone server, or server 300 may be a server in a virtual world computing system or in a cluster of two or more servers. In this example, server 300 is a server in a computing system for rendering and managing a virtual world.)
1. and processing the image based on the final scene category of the image. / 13. and processing the image based on the final scene category of the image. / 20. and processing the image based on the final scene category of the image. (Hatfield: [0146] Referring now to FIG. 6B a block diagram of the data flow 400 between various hardware and software components of a client/server environment implementing a virtual world is shown according to an illustrative embodiment. Virtual World client application 410 is a software application, executing on a client data processing system. Virtual World client application 410 allows for connection to, and interaction within a virtual world. In response to an indication from a user, virtual world client application 410 sends snapshot request 412 to virtual world server 414. Snapshot request 412 is a request for a graphical rendering of the viewable field of a particular user's avatar, as displayed by virtual world client application 410. The snapshot request can be a request for a static image, or a video of the rendering of the viewable field. Virtual world server 414 is a data processing system that executes software to enable a virtual world, such as virtual world server 300. Responsive to receiving snapshot request 412, virtual world server 414 forwards scene descriptions 416 to backend rendering system 418. Scene description 416 is a non-graphical representation of the viewable field of a particular user's avatar. Scene description 416 includes virtual world location information such as an exact camera location within the virtual world, and a camera angle vector. The exact camera location within the virtual world and the camera angle vector can be expressed as coordinates in a 3D coordinate system, such as for example in an <X, Y, Z> format, Scene description 416 can also include information regarding the format and delivery of the rendered snapshot, such as for example but not limited to, an image size, a file type format, delivery options, whether to include avatars within the snapshot, and desired fidelity/resolution of the snapshot. Scene description 416 going to backend rendering system 418 are a subset of the updates going to the client. Backend rendering system 418 is a graphics-processing unit capable of generating images and video of activity within a virtual world. Backend rendering system 418 is capable of rendering images and video of higher quality than local 3D rendering hardware and software running virtual world client application 410 because backend rendering system 418 utilizes images and textures that are themselves not used to create the virtual world as rendered by virtual world client application 410. Backend rendering system 418 produces high quality images by using higher quality models, shaders, and textures, or by implementing a ray tracing system. The models, shaders, textures, and ray tracing system utilized by backend rendering system 418 are not used in rendering the virtual world as is typically rendered by virtual world client application 410. In one illustrative embodiment, backend rendering system 418 implements high quality ray tracing effects using a Cell Broadband Engine Architecture based system. In one illustrative embodiment, backend rendering system 418 implements high quality models, shaders, and textures using traditional graphical processing unit hardware. Responsive to receiving scene description 416, backend rendering system 418 sends request 420 for high fidelity objects to visual database 422. Visual database 422 contains textures, geometry, and shaders that are needed to implement the virtual world. A scene description is downloaded onto backend rendering system 418, and the needed textures, geometry, and shaders are downloaded from the Visual database. Updates are then sent to backend rendering system 418 as the avatar moves in the scene. Backend rendering system 418 adjusts and moves the rendered avatars and objects accordingly. Visual database 422 stores geometric data, textures, and effects data associated with the form or shape of avatars and objects in the virtual world. Visual database 422 can be referenced by an Object avatar-rendering table in order to obtain geometric data, textures, and effects data to render the virtual world. Visual database 422 contains lightweight graphics 424 and high fidelity graphics 426. Lightweight graphics 424 include the 3D geometry, textures, shaders that are utilized by both virtual world server 414 and virtual world client application 410 to render the virtual world to the client. Lightweight graphics 424 are delivered to the virtual world server 414 in response to a request 428 for low fidelity objects sent from virtual world server 414 to visual database 422. High fidelity graphics 426 is a high fidelity version of geometry, textures, shaders that are utilized by backend rendering system 418 to create high fidelity snapshots and videos. Due to the bandwidth and processing requirements, high fidelity graphics 426 are not utilized by virtual world client application 410 in rendering the virtual world in real time. Backend rendering system 418 utilizes high fidelity graphics 426 to create high fidelity video and images 430. In one illustrative embodiment, backend rendering system 418 implements high quality ray tracing effects using a cell based system. In one illustrative embodiment, backend rendering System 418 implements high quality models, shaders, textures using traditional graphical processing unit hardware. In one illustrative embodiment, backend rendering system 418 implements high quality ray tracing effects using the Cell Broadband Engine Architecture based system. Backend rendering system 418 ray tracing determines the visibility of surfaces within an image by tracing imaginary rays of light from viewer's eye to the object. A separate ray is utilized for each resolved point, typically for each pixel of the display. For example, for a modest image resolution of 1064×768, a ray-tracer needs to trace 1064×768, or 817,152 separate rays for each displayed image. While the ray-traced image is very realistic due to the separate casting of each resolved point, the sheer number of rays that must be traced results in longer process times that are typically unsuitable for real time processing that must occur when implementing a virtual world. In one illustrative embodiment, backend rendering system 418 implements high quality ray tracing effects using a cell based system. A cell based system is a data processing system using the Cell Broadband Engine Architecture. The Cell Broadband Engine Architecture is a multi-core processor. The cell-based, system differs from traditional multi-core processors in that the cell based system does not consist of multiple copies of the same core. Instead, the cell based system utilizes heterogeneous processors, consisting of one PowerPC core and a plurality of synergistic co-processor elements. The synergistic co-processor elements are typically smaller than traditional cores and are designed for streaming workloads. The PowerPC core performs synchronization tasks and executes non-parallizeable code. The synergistic co-processor elements perform the balance of the work in a streaming manner. In one illustrative embodiment, backend rendering system 418 creates create high fidelity video and images 430 by sending scene updates to the backend rendering system as the client moves the avatar moves in the scene and objects move or change in the scene. The backend rendering system adjusts and moves the viewing position and rendered avatars and objects accordingly. In one illustrative embodiment, ray-traced images rendered by the Cell Broadband Engine Architecture based system of backend rendering system 418 are created at fast time intervals using scene updates. To create the illusion of smoother movement between the ray-traced key frames, the key frame animation software extrapolates in-between geometry and positions from scene updates to create a smoother animation. As the client navigates the virtual world, avatar and camera movement 432 are communicated to virtual world server 414. Additional scene descriptions 434 are passed to backend rendering system 418 and stored in scene update database 436 until additional scene descriptions 434 are utilized by backend rendering system 418. Scene update database 436 is a storage that holds scene descriptions, such as additional scene descriptions 434, until those scene descriptions are utilized by backend rendering system 418 to create high fidelity video and images 430. Thus, scene update database 436 acts as a buffer system, temporarily storing additional scene descriptions 434 for the backend rendering system 418. In this manner, backend rendering system 418 is able to function in a non-real time. Backend rendering system 418 is therefore able to create realistic images and videos without sacrificing resolution so that processing can be performed in real time. Once backend rendering system 418 creates high fidelity video and images 430, high fidelity video and images 430 are stored in video database 438 until high fidelity video and images 430 is delivered to a client system.)
It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify a method and system for controlling scene detection as disclosed by Folliot using machine learning models with the historical analysis of article provisioning presented by Hatfield, as they are both directed towards methods and systems for machine learning models for classification. The determination of obviousness is predicated upon the following findings: One skilled in the art would have been motivated to modify Folliot in order to leverage historical image data in the overall classification and machine learning models of Hatfield for controlling scene detection. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in the manner explained above using known engineering design, interface and/or programming techniques, without changing a “fundamental” operating principle of Folliot, while the teaching of Hatfield continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of leveraging historical previously recognized data in a machine learning model for scene classification. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Consider Claims 2 and 14.
The combination of Folliot and Hatfield teaches:
2. The system of claim 1, wherein the updating, based on the target scene category, at least a portion of multiple confidence level thresholds corresponding to the multiple scene categories includes: updating the at least a portion of the multiple confidence level thresholds by decreasing a confidence level threshold corresponding to the target scene category. / 14. The method of claim 13, wherein the updating, based on the target scene category, at least a portion of multiple confidence level thresholds corresponding to the multiple scene categories includes: updating the at least a portion of the multiple confidence level thresholds by decreasing a confidence level threshold corresponding to the target scene category. (Folliot: [0119] If the initial probability of confidence PC1 is not delivered by the discrimination algorithm ALC, it is considered that it has an arbitrary value, set for example, at 100%. In this case, the weighting by the first probability of transition TrPT1 is performed on the probability of confidence having the set value. [0120] The first probability of transition TrPT1 makes it possible to quantify the probability of the user changing from one scene to another. [0121] It is advantageously a parameter to be taken into account in scene detection, as certain transitions from one scene to another are unlikely, or even impossible. [0122] Thus, by taking the first probability of transition TrPT1 into account, the initial probability of confidence PC1 associated with the detected current scene is improved. Hatfield: [0085] Manager system 110 provisioning data predictive model 4102 in generating predicted provisioning data action decisions can assign confidence levels associated to the various predictions. The confidence level, in one example, can be dependent on trends exhibited by the training data and a volume of training data supporting certain prediction. [0086] Manager system 110 can scale scoring values under factor FV4 (predictive model factor) by reference to a decision data structure in which there are assigned action intensity values to particularly predicted provisioning actions. Scoring values assigned by manager system 110 under factor FV4 can be assigned according to the decision data structure of Table A, when predicted action confidence level intensity value are all considered in deriving an output scoring value under factor FV4.)
Consider Claims 3 and 15.
The combination of Folliot and Hatfield teaches:
3. The system of claim 1, wherein for each of the multiple scene categories, the confidence level of the image belonging to the scene category is generated using a scene recognition model, wherein the scene recognition model is configured to: extract a target feature from the image; determine a reference feature for the scene category; determine a similarity between the target feature of the image and the reference feature of the scene category; and designate the similarity between the target feature and the reference feature as the confidence level of the image belongs to the scene category. / 15. The method of claim 13, wherein for each of the multiple scene categories, the confidence level of the image belonging to the scene category is generated using a scene recognition model, wherein the scene recognition model is configured to: extract a target feature from the image; determine a reference feature for the scene category; determine a similarity between the target feature of the image and the reference feature of the scene category; and designate the similarity between the target feature and the reference feature as the confidence level of the image belongs to the scene category. (Hatfield: [0056] On receipt of the profile defining data sent at block 1401, manager system 110 can update user profiles of the various users in dependence on the received profile defining data and can store such updated profile data into users area 2121 of data repository 108. On receipt of the profile defining data sent by social media system 140 at send block 1401, manager system 110 can perform processing of the received profile defining data. The processing can include subjecting text based content of the various users to natural language processing to extract data on topics of interest to the various users as well as sentiment associated to such topics and at updating block 1103 can store such topic preferences and associated sentiments in respect to such topics in users area 2121 of data repository 108 associated to the various users subject to registration in response to receipt of registration data sent at send block 2301. On completion of updating at block 1103, manager system 110 can proceed to training block 1104. At training block 1104 manager system 110 can apply a next iteration of training data defined by the updated topic and sentiment profile data updated at block 1103 to clustering machine learning model 5102 described further herein in relation to FIGS. 5A and 5B. Manager system 110 can then loop back to iteratively perform the loop of blocks 1101-1104 while simultaneously branching to iteratively perform the loop of blocks 1105-1122 described further herein. Folliot: [0040] The classification algorithm may deliver the initial probability of confidence associated with the new current detected scene. [0041] If the classification algorithm does not deliver the initial probability of confidence associated with the new current detected scene, it is then possible to assign the new current detected scene a probability of confidence having an arbitrary value. [0042] If the value of the initial probability of confidence is not available, it is considered to have a value set at 100%. In this case, the weighting by the first probability of transition is performed on the probability having the value set at 100%. [0043] The processing operation of updating the initial probabilities of confidence is compatible with any type of classification algorithm, but also with any type of filtering processing operation performed on the basis of probabilities associated with the detected scenes by the classification algorithm. [0044] This processing operation of updating the initial probabilities of confidence is in particular compatible, when there is provision to assign an identifier to each reference scene, with a filtering processing operation on the current detected scenes carried out not only on the basis of the updated probability of confidence associated with each new current detected scene, but also on the basis of the identifier of each new current detected scene. (Each detected scene effectively has an identifier, since each detected scene is one of the reference scenes that each has an identifier.) [0045] A filtering processing operation of this type is described, for example, in the French patent application filed May 10, 2017 under no. 1754080 (counterpart U.S. application Ser. No. 15/924,608, filed Mar. 19, 2018). A ‘meta-filter’ that makes it possible to utilize the temporal correlation between two consecutive decisions has therefore been proposed in the French patent application. The meta-filter acts on successive decisions that are delivered by the classification algorithm, regardless of the type thereof and including a meta-classifier. These applications are incorporated herein by reference. [0046] Some Features Thereof Will be Recalled Here. [0047] The filtering processing operation is a sliding temporal filtering processing operation on these current detected scenes over a filtering window of size M, on the basis of the identifier of each new current detected scene taken into account in the window and of the updated probability of confidence associated with this new current detected scene.)
Consider Claims 4 and 16.
The combination of Folliot and Hatfield teaches:
4. The system of claim 3, wherein the determining, based on the multiple updated confidence level thresholds and confidence levels of the image corresponding to the multiple scene categories, a final scene category to which the image belongs includes: for each of the multiple scene categories corresponding to the multiple updated confidence level thresholds, determining a relationship between the confidence level of the image belonging to the scene category and an updated confidence level threshold of the scene category; determining, based on relationships corresponding to the multiple scene categories, a preliminary scene category to which the image belongs; and determining, based on the preliminary scene category of the image, the final scene category to which the image belongs. / 16. The method of claim 15, wherein the determining, based on the multiple updated confidence level thresholds and confidence levels of the image corresponding to the multiple scene categories, a final scene category to which the image belongs includes: for each of the multiple scene categories corresponding to the multiple updated confidence level thresholds, determining a relationship between the confidence level of the image belonging to the scene category and an updated confidence level threshold of the scene category; determining, based on relationships corresponding to the multiple scene categories, a preliminary scene category to which the image belongs; and determining, based on the preliminary scene category of the image, the final scene category to which the image belongs. (Folliot: [0064] Thus, if the ratio is lower than the threshold, which may be caused by a large variance, the degree of confidence is low, and it may then be decided to deliver either the scene that effectively has the highest total updated probability of confidence, but while assigning it an item of information characterizing it as ‘uncertain’, or the previous filtered detected scene in terms of time, as detected scene at the output of the filter. [0065] If, by contrast, the ratio is higher than or equal to this threshold, which may be caused by a small variance, the degree of confidence is high, and it may then be decided to deliver the scene that effectively has the highest total updated probability of confidence, which is possibly normalized, as detected scene at the output of the filter. Hatfield: [0101] In one example, manager system 110 can refrain from presenting any prompting data on a user interface main screen 3116 prompting provisioning data action unless the scoring value satisfies the threshold value. In this example, it can be seen if a virtual article encountered in virtual reality space has no corresponding physical world article, factor FV1 can be driven to a significantly low value, such that the threshold may not be triggered and no prompting provisioning data action may be presented on user interface main screen (rather, the prompting data 3102, 3112, 3122 may be accessible via menu 3118). In another example, prompting data for prompting provisioning can be presented on main screen 3116 irrespective of a scoring formula A scoring value. In another example of generating at block 1110, manager system 110 can provide the decision to auto provision, and not generate any prompting data where the scoring value determined under scoring formula A satisfies a scoring threshold value for auto provisioning. Embodiments herein recognize that certain historical behaviors can be so indicative of provisioning of a user deciding to provision a virtual world article that prompting for such provisioning can be avoided and manager system 110 can proceed directly to reprovisioning the virtual world article without first prompting for the reprovisioning.)
Consider Claims 5 and 17.
The combination of Folliot and Hatfield teaches:
5. The system of claim 4, wherein the determining, based on the preliminary scene category of the image, the final scene category to which the image belongs includes: determining whether the preliminary scene category of the image and the initial scene categories of the multiple historical images are consistent in response to the initial scene category of the image and the initial scene categories of the multiple historical images being consistent, designating the preliminary scene category of the image as an updated scene category of the image, or in response to an inconsistency of the preliminary scene category of the image and the initial scene categories of the multiple historical images, determining the target scene category as an updated scene category of the image; and determining the final scene category of the image based on the updated scene category. / 17. The method of claim 16, wherein the determining, based on the preliminary scene category of the image, the final scene category to which the image belongs includes: determining whether the preliminary scene category of the image and the initial scene categories of the multiple historical images are consistent; in response to the initial scene category of the image and the initial scene categories of the multiple historical images being consistent, designating the preliminary scene category of the image as an updated scene category of the image, or in response to an inconsistency of the preliminary scene category of the image and the initial scene categories of the multiple historical images, determining the target scene category as an updated scene category of the image; and determining the final scene category of the image based on the updated scene category. (Hatfield: [0056] On receipt of the profile defining data sent at block 1401, manager system 110 can update user profiles of the various users in dependence on the received profile defining data and can store such updated profile data into users area 2121 of data repository 108. On receipt of the profile defining data sent by social media system 140 at send block 1401, manager system 110 can perform processing of the received profile defining data. The processing can include subjecting text based content of the various users to natural language processing to extract data on topics of interest to the various users as well as sentiment associated to such topics and at updating block 1103 can store such topic preferences and associated sentiments in respect to such topics in users area 2121 of data repository 108 associated to the various users subject to registration in response to receipt of registration data sent at send block 2301. On completion of updating at block 1103, manager system 110 can proceed to training block 1104. At training block 1104 manager system 110 can apply a next iteration of training data defined by the updated topic and sentiment profile data updated at block 1103 to clustering machine learning model 5102 described further herein in relation to FIGS. 5A and 5B. Manager system 110 can then loop back to iteratively perform the loop of blocks 1101-1104 while simultaneously branching to iteratively perform the loop of blocks 1105-1122 described further herein. [0063] On completion of recording block 2304, UE device 1301 can proceed to block 2305. At block 2305, UE device 1301 can ascertain whether the user has specified a virtual reality session request. Where a user has not requested a virtual reality session request, UE device 1301 can iteratively perform the loop of blocks 2303-2305 recording data for articles detected until a time that a VR session request is detected at block 2305. Recorded article classification data recorded at block 2304 can include a session stamp specifying the physical world session in which the detection occurred and a time stamp for the article detecting. It will be seen that a same article can be detected iteratively at iterations of block 2303. Folliot: [0040] The classification algorithm may deliver the initial probability of confidence associated with the new current detected scene. [0041] If the classification algorithm does not deliver the initial probability of confidence associated with the new current detected scene, it is then possible to assign the new current detected scene a probability of confidence having an arbitrary value. [0042] If the value of the initial probability of confidence is not available, it is considered to have a value set at 100%. In this case, the weighting by the first probability of transition is performed on the probability having the value set at 100%. [0043] The processing operation of updating the initial probabilities of confidence is compatible with any type of classification algorithm, but also with any type of filtering processing operation performed on the basis of probabilities associated with the detected scenes by the classification algorithm. [0044] This processing operation of updating the initial probabilities of confidence is in particular compatible, when there is provision to assign an identifier to each reference scene, with a filtering processing operation on the current detected scenes carried out not only on the basis of the updated probability of confidence associated with each new current detected scene, but also on the basis of the identifier of each new current detected scene. (Each detected scene effectively has an identifier, since each detected scene is one of the reference scenes that each has an identifier.) [0045] A filtering processing operation of this type is described, for example, in the French patent application filed May 10, 2017 under no. 1754080 (counterpart U.S. application Ser. No. 15/924,608, filed Mar. 19, 2018). A ‘meta-filter’ that makes it possible to utilize the temporal correlation between two consecutive decisions has therefore been proposed in the French patent application. The meta-filter acts on successive decisions that are delivered by the classification algorithm, regardless of the type thereof and including a meta-classifier. These applications are incorporated herein by reference. [0046] Some Features Thereof Will be Recalled Here. [0047] The filtering processing operation is a sliding temporal filtering processing operation on these current detected scenes over a filtering window of size M, on the basis of the identifier of each new current detected scene taken into account in the window and of the updated probability of confidence associated with this new current detected scene. [0070] According to one embodiment, the processor is configured to update the first probability of transition if, during a given time interval, at least one transition from the first scene to the second scene is performed, and this updating comprises calculation of a second probability of transition for each transition from the first scene to each of the possible second scenes from among the set of possible reference scenes.)
Consider Claims 6 and 18.
The combination of Folliot and Hatfield teaches:
6. The system of claim 5, wherein the determining the final scene category of the image based on the updated scene category includes: obtaining a count of scene category switches within a preset time period, wherein the preset time period is a time period during which the image and the multiple historical images are captured, and two adjacent scene categories of two continuous images being not consistent is determined as one scene category switch; and in response to the count of scene category switches exceeding a preset threshold, determining the preliminary scene category of the image as the final scene category of the image./ 18. The method of claim 17, wherein the determining the final scene category of the image based on the updated scene category includes:obtaining a count of scene category switches within a preset time period, wherein the preset time period is a time period during which the image and the multiple historical images are captured, and two adjacent scene categories of two continuous images being not consistent is determined as one scene category switch; and in response to the count of scene category switches exceeding a preset threshold, determining the preliminary scene category of the image as the final scene category of the image. (Hatfield: [0056] On receipt of the profile defining data sent at block 1401, manager system 110 can update user profiles of the various users in dependence on the received profile defining data and can store such updated profile data into users area 2121 of data repository 108. On receipt of the profile defining data sent by social media system 140 at send block 1401, manager system 110 can perform processing of the received profile defining data. The processing can include subjecting text based content of the various users to natural language processing to extract data on topics of interest to the various users as well as sentiment associated to such topics and at updating block 1103 can store such topic preferences and associated sentiments in respect to such topics in users area 2121 of data repository 108 associated to the various users subject to registration in response to receipt of registration data sent at send block 2301. On completion of updating at block 1103, manager system 110 can proceed to training block 1104. At training block 1104 manager system 110 can apply a next iteration of training data defined by the updated topic and sentiment profile data updated at block 1103 to clustering machine learning model 5102 described further herein in relation to FIGS. 5A and 5B. Manager system 110 can then loop back to iteratively perform the loop of blocks 1101-1104 while simultaneously branching to iteratively perform the loop of blocks 1105-1122 described further herein. [0063] On completion of recording block 2304, UE device 1301 can proceed to block 2305. At block 2305, UE device 1301 can ascertain whether the user has specified a virtual reality session request. Where a user has not requested a virtual reality session request, UE device 1301 can iteratively perform the loop of blocks 2303-2305 recording data for articles detected until a time that a VR session request is detected at block 2305. Recorded article classification data recorded at block 2304 can include a session stamp specifying the physical world session in which the detection occurred and a time stamp for the article detecting. It will be seen that a same article can be detected iteratively at iterations of block 2303. [0101] In one example, manager system 110 can refrain from presenting any prompting data on a user interface main screen 3116 prompting provisioning data action unless the scoring value satisfies the threshold value. In this example, it can be seen if a virtual article encountered in virtual reality space has no corresponding physical world article, factor FV1 can be driven to a significantly low value, such that the threshold may not be triggered and no prompting provisioning data action may be presented on user interface main screen (rather, the prompting data 3102, 3112, 3122 may be accessible via menu 3118). In another example, prompting data for prompting provisioning can be presented on main screen 3116 irrespective of a scoring formula A scoring value. In another example of generating at block 1110, manager system 110 can provide the decision to auto provision, and not generate any prompting data where the scoring value determined under scoring formula A satisfies a scoring threshold value for auto provisioning. Embodiments herein recognize that certain historical behaviors can be so indicative of provisioning of a user deciding to provision a virtual world article that prompting for such provisioning can be avoided and manager system 110 can proceed directly to reprovisioning the virtual world article without first prompting for the reprovisioning. Folliot: [0040] The classification algorithm may deliver the initial probability of confidence associated with the new current detected scene. [0041] If the classification algorithm does not deliver the initial probability of confidence associated with the new current detected scene, it is then possible to assign the new current detected scene a probability of confidence having an arbitrary value. [0042] If the value of the initial probability of confidence is not available, it is considered to have a value set at 100%. In this case, the weighting by the first probability of transition is performed on the probability having the value set at 100%. [0043] The processing operation of updating the initial probabilities of confidence is compatible with any type of classification algorithm, but also with any type of filtering processing operation performed on the basis of probabilities associated with the detected scenes by the classification algorithm. [0044] This processing operation of updating the initial probabilities of confidence is in particular compatible, when there is provision to assign an identifier to each reference scene, with a filtering processing operation on the current detected scenes carried out not only on the basis of the updated probability of confidence associated with each new current detected scene, but also on the basis of the identifier of each new current detected scene. (Each detected scene effectively has an identifier, since each detected scene is one of the reference scenes that each has an identifier.) [0045] A filtering processing operation of this type is described, for example, in the French patent application filed May 10, 2017 under no. 1754080 (counterpart U.S. application Ser. No. 15/924,608, filed Mar. 19, 2018). A ‘meta-filter’ that makes it possible to utilize the temporal correlation between two consecutive decisions has therefore been proposed in the French patent application. The meta-filter acts on successive decisions that are delivered by the classification algorithm, regardless of the type thereof and including a meta-classifier. These applications are incorporated herein by reference. [0046] Some Features Thereof Will be Recalled Here. [0047] The filtering processing operation is a sliding temporal filtering processing operation on these current detected scenes over a filtering window of size M, on the basis of the identifier of each new current detected scene taken into account in the window and of the updated probability of confidence associated with this new current detected scene. [0064] Thus, if the ratio is lower than the threshold, which may be caused by a large variance, the degree of confidence is low, and it may then be decided to deliver either the scene that effectively has the highest total updated probability of confidence, but while assigning it an item of information characterizing it as ‘uncertain’, or the previous filtered detected scene in terms of time, as detected scene at the output of the filter. [0065] If, by contrast, the ratio is higher than or equal to this threshold, which may be caused by a small variance, the degree of confidence is high, and it may then be decided to deliver the scene that effectively has the highest total updated probability of confidence, which is possibly normalized, as detected scene at the output of the filter. [0070] According to one embodiment, the processor is configured to update the first probability of transition if, during a given time interval, at least one transition from the first scene to the second scene is performed, and this updating comprises calculation of a second probability of transition for each transition from the first scene to each of the possible second scenes from among the set of possible reference scenes.)
Consider Claims 7 and 19.
The combination of Folliot and Hatfield teaches:
7. The system of claim 6, wherein the operations further include: in response to the count of scene category switches not exceeding the preset threshold, determining the updated scene category as the final scene category of the image. / 19. The method of claim 18, wherein the operations further include: in response to the count of scene category switches not exceeding the preset threshold, determining the updated scene category as the final scene category of the image. (Folliot: [0064] Thus, if the ratio is lower than the threshold, which may be caused by a large variance, the degree of confidence is low, and it may then be decided to deliver either the scene that effectively has the highest total updated probability of confidence, but while assigning it an item of information characterizing it as ‘uncertain’, or the previous filtered detected scene in terms of time, as detected scene at the output of the filter. [0065] If, by contrast, the ratio is higher than or equal to this threshold, which may be caused by a small variance, the degree of confidence is high, and it may then be decided to deliver the scene that effectively has the highest total updated probability of confidence, which is possibly normalized, as detected scene at the output of the filter. Hatfield: [0101] In one example, manager system 110 can refrain from presenting any prompting data on a user interface main screen 3116 prompting provisioning data action unless the scoring value satisfies the threshold value. In this example, it can be seen if a virtual article encountered in virtual reality space has no corresponding physical world article, factor FV1 can be driven to a significantly low value, such that the threshold may not be triggered and no prompting provisioning data action may be presented on user interface main screen (rather, the prompting data 3102, 3112, 3122 may be accessible via menu 3118). In another example, prompting data for prompting provisioning can be presented on main screen 3116 irrespective of a scoring formula A scoring value. In another example of generating at block 1110, manager system 110 can provide the decision to auto provision, and not generate any prompting data where the scoring value determined under scoring formula A satisfies a scoring threshold value for auto provisioning. Embodiments herein recognize that certain historical behaviors can be so indicative of provisioning of a user deciding to provision a virtual world article that prompting for such provisioning can be avoided and manager system 110 can proceed directly to reprovisioning the virtual world article without first prompting for the reprovisioning.)
Consider Claim 8.
The combination of Folliot and Hatfield teaches:
8. The system of claim 1, wherein different scene categories correspond to different image processing techniques, and the processing the image based on the final scene category of the image includes: processing the image using an image processing technique corresponding to the final scene category of the image. (Hatfield: [0056] On receipt of the profile defining data sent at block 1401, manager system 110 can update user profiles of the various users in dependence on the received profile defining data and can store such updated profile data into users area 2121 of data repository 108. On receipt of the profile defining data sent by social media system 140 at send block 1401, manager system 110 can perform processing of the received profile defining data. The processing can include subjecting text based content of the various users to natural language processing to extract data on topics of interest to the various users as well as sentiment associated to such topics and at updating block 1103 can store such topic preferences and associated sentiments in respect to such topics in users area 2121 of data repository 108 associated to the various users subject to registration in response to receipt of registration data sent at send block 2301. On completion of updating at block 1103, manager system 110 can proceed to training block 1104. At training block 1104 manager system 110 can apply a next iteration of training data defined by the updated topic and sentiment profile data updated at block 1103 to clustering machine learning model 5102 described further herein in relation to FIGS. 5A and 5B. Manager system 110 can then loop back to iteratively perform the loop of blocks 1101-1104 while simultaneously branching to iteratively perform the loop of blocks 1105-1122 described further herein. [0063] On completion of recording block 2304, UE device 1301 can proceed to block 2305. At block 2305, UE device 1301 can ascertain whether the user has specified a virtual reality session request. Where a user has not requested a virtual reality session request, UE device 1301 can iteratively perform the loop of blocks 2303-2305 recording data for articles detected until a time that a VR session request is detected at block 2305. Recorded article classification data recorded at block 2304 can include a session stamp specifying the physical world session in which the detection occurred and a time stamp for the article detecting. It will be seen that a same article can be detected iteratively at iterations of block 2303. [0101] In one example, manager system 110 can refrain from presenting any prompting data on a user interface main screen 3116 prompting provisioning data action unless the scoring value satisfies the threshold value. In this example, it can be seen if a virtual article encountered in virtual reality space has no corresponding physical world article, factor FV1 can be driven to a significantly low value, such that the threshold may not be triggered and no prompting provisioning data action may be presented on user interface main screen (rather, the prompting data 3102, 3112, 3122 may be accessible via menu 3118). In another example, prompting data for prompting provisioning can be presented on main screen 3116 irrespective of a scoring formula A scoring value. In another example of generating at block 1110, manager system 110 can provide the decision to auto provision, and not generate any prompting data where the scoring value determined under scoring formula A satisfies a scoring threshold value for auto provisioning. Embodiments herein recognize that certain historical behaviors can be so indicative of provisioning of a user deciding to provision a virtual world article that prompting for such provisioning can be avoided and manager system 110 can proceed directly to reprovisioning the virtual world article without first prompting for the reprovisioning. Folliot: [0040] The classification algorithm may deliver the initial probability of confidence associated with the new current detected scene. [0041] If the classification algorithm does not deliver the initial probability of confidence associated with the new current detected scene, it is then possible to assign the new current detected scene a probability of confidence having an arbitrary value. [0042] If the value of the initial probability of confidence is not available, it is considered to have a value set at 100%. In this case, the weighting by the first probability of transition is performed on the probability having the value set at 100%. [0043] The processing operation of updating the initial probabilities of confidence is compatible with any type of classification algorithm, but also with any type of filtering processing operation performed on the basis of probabilities associated with the detected scenes by the classification algorithm. [0044] This processing operation of updating the initial probabilities of confidence is in particular compatible, when there is provision to assign an identifier to each reference scene, with a filtering processing operation on the current detected scenes carried out not only on the basis of the updated probability of confidence associated with each new current detected scene, but also on the basis of the identifier of each new current detected scene. (Each detected scene effectively has an identifier, since each detected scene is one of the reference scenes that each has an identifier.) [0045] A filtering processing operation of this type is described, for example, in the French patent application filed May 10, 2017 under no. 1754080 (counterpart U.S. application Ser. No. 15/924,608, filed Mar. 19, 2018). A ‘meta-filter’ that makes it possible to utilize the temporal correlation between two consecutive decisions has therefore been proposed in the French patent application. The meta-filter acts on successive decisions that are delivered by the classification algorithm, regardless of the type thereof and including a meta-classifier. These applications are incorporated herein by reference. [0046] Some Features Thereof Will be Recalled Here. [0047] The filtering processing operation is a sliding temporal filtering processing operation on these current detected scenes over a filtering window of size M, on the basis of the identifier of each new current detected scene taken into account in the window and of the updated probability of confidence associated with this new current detected scene. [0064] Thus, if the ratio is lower than the threshold, which may be caused by a large variance, the degree of confidence is low, and it may then be decided to deliver either the scene that effectively has the highest total updated probability of confidence, but while assigning it an item of information characterizing it as ‘uncertain’, or the previous filtered detected scene in terms of time, as detected scene at the output of the filter. [0065] If, by contrast, the ratio is higher than or equal to this threshold, which may be caused by a small variance, the degree of confidence is high, and it may then be decided to deliver the scene that effectively has the highest total updated probability of confidence, which is possibly normalized, as detected scene at the output of the filter. [0070] According to one embodiment, the processor is configured to update the first probability of transition if, during a given time interval, at least one transition from the first scene to the second scene is performed, and this updating comprises calculation of a second probability of transition for each transition from the first scene to each of the possible second scenes from among the set of possible reference scenes.)
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Folliot et al. (US PGPub US 20190287009 A1, hereby referred to as “Folliot”), in view of Denninghoff et al. (US Patent 2,222,222), hereby referred to as “Denninghoff” further in view of Di Pietro (US Patent US 10,317,240 B1, hereby referred to as “Di Pietro”).
Consider Claim 9.
The combination of Folliot and Hatfield teaches:
9. The system of claim 1, wherein the processing the image based on the final scene category of the image includes: determining whether the final scene category is a preset scene category, (Hatfield: [0146] Referring now to FIG. 6B a block diagram of the data flow 400 between various hardware and software components of a client/server environment implementing a virtual world is shown according to an illustrative embodiment. Virtual World client application 410 is a software application, executing on a client data processing system. Virtual World client application 410 allows for connection to, and interaction within a virtual world. In response to an indication from a user, virtual world client application 410 sends snapshot request 412 to virtual world server 414. Snapshot request 412 is a request for a graphical rendering of the viewable field of a particular user's avatar, as displayed by virtual world client application 410. The snapshot request can be a request for a static image, or a video of the rendering of the viewable field. Virtual world server 414 is a data processing system that executes software to enable a virtual world, such as virtual world server 300. Responsive to receiving snapshot request 412, virtual world server 414 forwards scene descriptions 416 to backend rendering system 418. Scene description 416 is a non-graphical representation of the viewable field of a particular user's avatar. Scene description 416 includes virtual world location information such as an exact camera location within the virtual world, and a camera angle vector. The exact camera location within the virtual world and the camera angle vector can be expressed as coordinates in a 3D coordinate system, such as for example in an <X, Y, Z> format, Scene description 416 can also include information regarding the format and delivery of the rendered snapshot, such as for example but not limited to, an image size, a file type format, delivery options, whether to include avatars within the snapshot, and desired fidelity/resolution of the snapshot. Scene description 416 going to backend rendering system 418 are a subset of the updates going to the client. Backend rendering system 418 is a graphics-processing unit capable of generating images and video of activity within a virtual world. Backend rendering system 418 is capable of rendering images and video of higher quality than local 3D rendering hardware and software running virtual world client application 410 because backend rendering system 418 utilizes images and textures that are themselves not used to create the virtual world as rendered by virtual world client application 410. Backend rendering system 418 produces high quality images by using higher quality models, shaders, and textures, or by implementing a ray tracing system. The models, shaders, textures, and ray tracing system utilized by backend rendering system 418 are not used in rendering the virtual world as is typically rendered by virtual world client application 410. In one illustrative embodiment, backend rendering system 418 implements high quality ray tracing effects using a Cell Broadband Engine Architecture based system. In one illustrative embodiment, backend rendering system 418 implements high quality models, shaders, and textures using traditional graphical processing unit hardware. Responsive to receiving scene description 416, backend rendering system 418 sends request 420 for high fidelity objects to visual database 422. Visual database 422 contains textures, geometry, and shaders that are needed to implement the virtual world. A scene description is downloaded onto backend rendering system 418, and the needed textures, geometry, and shaders are downloaded from the Visual database. Updates are then sent to backend rendering system 418 as the avatar moves in the scene. Backend rendering system 418 adjusts and moves the rendered avatars and objects accordingly. Visual database 422 stores geometric data, textures, and effects data associated with the form or shape of avatars and objects in the virtual world. Visual database 422 can be referenced by an Object avatar-rendering table in order to obtain geometric data, textures, and effects data to render the virtual world. Visual database 422 contains lightweight graphics 424 and high fidelity graphics 426. Lightweight graphics 424 include the 3D geometry, textures, shaders that are utilized by both virtual world server 414 and virtual world client application 410 to render the virtual world to the client. Lightweight graphics 424 are delivered to the virtual world server 414 in response to a request 428 for low fidelity objects sent from virtual world server 414 to visual database 422. High fidelity graphics 426 is a high fidelity version of geometry, textures, shaders that are utilized by backend rendering system 418 to create high fidelity snapshots and videos. Due to the bandwidth and processing requirements, high fidelity graphics 426 are not utilized by virtual world client application 410 in rendering the virtual world in real time. Backend rendering system 418 utilizes high fidelity graphics 426 to create high fidelity video and images 430. In one illustrative embodiment, backend rendering system 418 implements high quality ray tracing effects using a cell based system. In one illustrative embodiment, backend rendering System 418 implements high quality models, shaders, textures using traditional graphical processing unit hardware. In one illustrative embodiment, backend rendering system 418 implements high quality ray tracing effects using the Cell Broadband Engine Architecture based system. Backend rendering system 418 ray tracing determines the visibility of surfaces within an image by tracing imaginary rays of light from viewer's eye to the object. A separate ray is utilized for each resolved point, typically for each pixel of the display. For example, for a modest image resolution of 1064×768, a ray-tracer needs to trace 1064×768, or 817,152 separate rays for each displayed image. While the ray-traced image is very realistic due to the separate casting of each resolved point, the sheer number of rays that must be traced results in longer process times that are typically unsuitable for real time processing that must occur when implementing a virtual world. In one illustrative embodiment, backend rendering system 418 implements high quality ray tracing effects using a cell based system. A cell based system is a data processing system using the Cell Broadband Engine Architecture. The Cell Broadband Engine Architecture is a multi-core processor. The cell-based, system differs from traditional multi-core processors in that the cell based system does not consist of multiple copies of the same core. Instead, the cell based system utilizes heterogeneous processors, consisting of one PowerPC core and a plurality of synergistic co-processor elements. The synergistic co-processor elements are typically smaller than traditional cores and are designed for streaming workloads. The PowerPC core performs synchronization tasks and executes non-parallizeable code. The synergistic co-processor elements perform the balance of the work in a streaming manner. In one illustrative embodiment, backend rendering system 418 creates create high fidelity video and images 430 by sending scene updates to the backend rendering system as the client moves the avatar moves in the scene and objects move or change in the scene. The backend rendering system adjusts and moves the viewing position and rendered avatars and objects accordingly. In one illustrative embodiment, ray-traced images rendered by the Cell Broadband Engine Architecture based system of backend rendering system 418 are created at fast time intervals using scene updates. To create the illusion of smoother movement between the ray-traced key frames, the key frame animation software extrapolates in-between geometry and positions from scene updates to create a smoother animation. As the client navigates the virtual world, avatar and camera movement 432 are communicated to virtual world server 414. Additional scene descriptions 434 are passed to backend rendering system 418 and stored in scene update database 436 until additional scene descriptions 434 are utilized by backend rendering system 418. Scene update database 436 is a storage that holds scene descriptions, such as additional scene descriptions 434, until those scene descriptions are utilized by backend rendering system 418 to create high fidelity video and images 430. Thus, scene update database 436 acts as a buffer system, temporarily storing additional scene descriptions 434 for the backend rendering system 418. In this manner, backend rendering system 418 is able to function in a non-real time. Backend rendering system 418 is therefore able to create realistic images and videos without sacrificing resolution so that processing can be performed in real time. Once backend rendering system 418 creates high fidelity video and images 430, high fidelity video and images 430 are stored in video database 438 until high fidelity video and images 430 is delivered to a client system. Folliott: [0090] The decision tree ALC acts on a vector of attributes. The tree comprises a series of nodes. Each node is assigned to test an attribute. [0091] Two branches leave a node. [0092] Moreover, the output of the tree includes leaves corresponding to reference scenes that the device APP is supposed to detect. [0093] These reference scenes may be for example, without limitation, ‘BOAT’, ‘PLANE’, ‘VEHICLE’, ‘WALKING’ scenes that are representative for example, of the environment in which the device APP, in this case the telephone, may be located. [0094] The detector MDET also includes an acquisition circuit ACQ that is configured to acquire current values of the attributes on the basis of the measurement data originating from the sensors. [0095] The detector MDET includes a command circuit MCM that is configured to activate the software module ALC with the current values of the attributes so as to take a path within the decision tree and obtain, at the output of the path, one scene from among the set of reference scenes, this obtained scene forming the detected scene. [0096] Moreover, the discrimination algorithm also delivers an initial probability of confidence associated with the detected scene, which will be stored in a memory MM2. [0097] In this respect, the initial probability of confidence associated with the detected scene is formulated following the detection of the detected scene and on the basis of the knowledge of this detected scene, in particular by making an additional journey on the path with the knowledge, at each node, of the detected scene. [0098] Other classification algorithms or classifiers that are different from the decision trees and well known to those skilled in the art exist for detecting scenes. Mention may be made in particular of a neural network algorithm that is known to those skilled in the art and for which reference may be made for example, for all useful purposes, to the work by Martin T. Hagan and Howard B. Demuth entitled Neural Network Design (2nd Edition), 1 Sep. 2014, or else an algorithm known to those skilled in the art under the name GMM (‘Gaussian Mixture Model’), those skilled in the art being able to refer for example, for all useful purposes, to the tutorial slides by Andrew Moore entitled ‘Clustering with Gaussian Mixtures’, available on the website https://www.autonlab.org/tutorials/gmm.html. [0099] These two algorithms are also configured to deliver an initial probability of confidence for each detected scene.
[0100] The device APP also includes a processor MDC that is configured to update the initial probability of confidence. [0101] The processor MDC comprises a counter MDC1, the role of which will be detailed below, and an arithmetic logic unit (ALU) MDC2 configured to carry out arithmetic operations and comparisons. [0102] The operations carried out by the arithmetic logic unit MDC2 will be detailed in FIGS. 4 and 5. [0103] The device APP also includes a block BLC able to interact with the detector MDET in order to process the detected scene and transmit the information via the antenna ANT of the device.)
The combination of Folliot and Hatfield does not teach: “the preset scene category including at least one of a face recognition category or a license plate recognition category”
Di Pietro teaches:
9. The system of claim 1, wherein the processing the image based on the final scene category of the image includes: determining whether the final scene category is a preset scene category, the preset scene category including at least one of a face recognition category or a license plate recognition category (Di Pietro: abstract, Travel data may be gathered and published on a global map for use by various entities. The travel data may be captured by one or more vehicle computing devices, such as those associated with the operation and/or control of a vehicle. The vehicle computing devices may receive input from one or more sensors, and may identify and/or classify an event and/or areas of interest based on the input. The vehicle computing devices may then send travel data to the global map server for publication. Additionally or alternatively, the travel data may be captured by one or more user devices. The user devices may be configured to receive travel data, such as via a user interface, and to send the travel data to the global map server for publication. The global map server may consolidate and publish the travel data on a global map. column 3 lines 64-67, column 4 lines 1-3, In some examples, the global map server may anonymize the travel data. Anonymizing the data may include removing (e.g., blur, black out, etc.) sensitive data, such as license plate numbers or other identifying information (e.g., faces, distinguishing tattoos, scars, etc.). Additionally, the anonymizing may include removing offensive images from the global map. For example, an image of a motorcycle accident may include an accident victim with a compound fracture. The global map server may blur or otherwise remove the accident victim from the image, to account for viewer sensitivities. Column 10 lines 45-67, column 11 lines 1-23,)
in response to determining that the final scene category is the preset scene category, obtaining a target region within the image; determining, based on gray information of the target region, a current brightness of the target region; and processing the image based on the current brightness of the target region.(Di Pietro: Column 10 lines 45-67, column 11 lines 1-23, In various examples, the global map server 102 and/or the computing device 104 may be configured to anonymize the travel data 114. Anonymizing the data may include filtering (e.g., removing, blurring to render unrecognizable, etc.) sensitive data from the travel data 114. Sensitive data may include data which can be used to identify an individual, such as license plate numbers, bumper stickers, physical characteristics (e.g., a face, a distinctive tattoo, etc.), voices, or other data. Additionally, sensitive data may include offensive language, and/or other undesirable images or audio. In various examples, the global map server 102 and/or computing device 104 may be configured to recognize the sensitive data for anonymity filtering. In some examples, the recognition may be by object, text, and/or speech recognition. In various examples, the global map server 102 may apply machine learning techniques to identify sensitive data in travel data 114. In such examples, the global map server 102 may be trained with a dataset to recognize license plates, names, words, phrases, and/or other sensitive data. Additionally, the anonymizing may include removing offensive images from the travel data 114. For example, an image of a motorcycle accident may include an accident victim with a compound fracture. The global map server 102 and/or computing device 104 may blur or otherwise remove the accident victim from the image, to account for viewer sensitivities.)
It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify a machine learning method and system for controlling scene detection using historical image analytics as disclosed by the combination of Folliot and Hatfield, and apply it to other fields of image analysis such as facial and license plate detection as suggested by De Pietro. The determination of obviousness is predicated upon the following findings: One skilled in the art would have been motivated to modify the combination of Folliot and Hatfield and apply the overall classification and machine learning models to other realms of image analysis such as facial detection and license plate recognition. Both would lend themselves for combination with a predicted and reasonable expectation of success, and as the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in the manner explained above using known engineering design, interface and programming techniques, without changing a “fundamental” operating principle of the combination of Folliot and Hatfield, while the teaching of De Pietro continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of ensuring an improved machine learning model for scene classification to apply it to facial detection and license plate recognition. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Allowable Subject Matter
Claims 10-12 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter. The prior art fails to teach the method of Claim 10 or the method of Claim 12 which specifically comprises the following features in combination with other recited limitations. Claim 11 is dependent upon Claim 10. These features are supported in the foreign application for CN 201910568901.3 filed on June 27, 2019 and as such these features are not obviated by the prior art
10. The system of claim 9, wherein the image is a human image that includes a human and the target region is a facial region, and the obtaining a target region within the image includes: identifying an initial region including the target region from the image based on an object detection model, the object detection model being a deep learning model, wherein the initial region includes multiple objects, the multiple objects include the human, and the identifying the initial region from the image includes: extracting one or more features of each object in the image; assigning a score for the each object based on the object detection model; in response to the score of an object exceeds a threshold, determining a region of the image where the object is located as the initial region; trimming the initial region according to a predetermined aspect ratio to obtain a trimmed initial region; and determining the target region by performing a G-channel down sampling on the trimmed initial region.
11. The system of claim 10, wherein when the object is the human, and the trimming the initial region includes: when the initial region includes a whole body of the human, trimming the initial region by applying a three-step trimming strategy, the three-step trimming strategy including: obtaining a trimmed region by trimming a lower part of the initial region, the trimmed region including an upper part of the body of the human; trimming a portion of an upper part of the trimmed region and a portion of a lower part of the trimmed region; and symmetrically trimming a portion of a left side and a right side of the trimmed region to obtain the trimmed initial region; when the initial region includes the upper part of the body of the human, trimming the initial region by applying a two-step trimming strategy, the two-step trimming strategy including: trimming a portion of an upper part of the initial region and a portion of the lower part of the initial region; and symmetrically trimming a portion of a left side and a right side of the initial region to obtain the trimmed initial region; and when the initial region includes a lower part of the body of the human, removing the initial region.
12. The system of claim 9, wherein the processing the image based on the current brightness of the target region includes: in response to determining that the current brightness of the target region is less than a minimum value of a predetermined range, increasing the current brightness of the target region, the more the current brightness is less than the minimum value of the predetermined range, the greater the increasing of the current brightness is; Andin response to determining that the current brightness of the target region is greater than a maximum value of the predetermined range, decreasing the current brightness of the target region, the more the current brightness is greater than the maximum value of the predetermined range, the greater the decreasing of the current brightness is.
Conclusion
The prior art made of record in form PTO-892 and not relied upon is considered pertinent to applicant's disclosure.
PNG
media_image1.png
172
934
media_image1.png
Greyscale
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TAHMINA ANSARI whose telephone number is 571-270-3379. The examiner can normally be reached on IFP Flex - Monday through Friday 9 to 5.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O’NEAL MISTRY can be reached on 313-446-4912. The fax phone numbers for the organization where this application or proceeding is assigned are 571-273-8300 for regular communications and 571-273-8300 for After Final communications. TC 2600’s customer service number is 571-272-2600.
Any inquiry of a general nature or relating to the status of this application or proceeding should be directed to the receptionist whose telephone number is 571-272-2600.
2674
/Tahmina Ansari/
October 8, 2026
/TAHMINA N ANSARI/Primary Examiner, Art Unit 2674
